Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Programming

Tablet Optimization Tips in the Google Play Developer Console

Android Developers Blog - Thu, 04/25/2013 - 18:03

Posted by Ellie Powers, Google Play team

Last week we updated our guidelines for making great tablet apps and added the ability to upload tablet screenshots that are shown preferentially in Google Play to users on those devices. Today we’re introducing a new Optimization Tips page in the Google Play Developer Console that lets you quickly see how your app is doing against basic guidelines for tablet app distribution and quality.

When you upload an app, the Developer Console now runs a series of checks to verify basic criteria from the Tablet App Quality Checklist and shows you any issues it finds in the Optimization Tips page.

If you’re developing for tablets, make sure to visit your Optimization Tips page to ensure that your app is delivering a great tablet experience. If there are any issues listed, we recommend addressing them in your app as soon as possible and uploading a new binary for distribution, if needed.

For ideas on how to design and build a great tablet app, including details on how to address issues listed in your Optimization Tips page, check out the Tablet App Quality Checklist. Remember that a great tablet experience goes well beyond these basic checks. Keep working to bring your tablet users the most polished UI and richest content possible.

Join the discussion on

+Android Developers
Categories: Programming

The Key to Agility: Breaking Things Down

If you find you can't keep up with the world around you, then break things down.  Breaking things down is the key to finishing faster.

Breaking things down is also the key to agility.

One of the toughest project management lessons I had to learn was breaking things down into more modular chunks.   When I took on a project, my goal was to make big things happen and change the world. 

After all, go big or go home, right?

The problem is you run out of time, or you run out of budget.  You even run out of oomph.  So the worst way to make things happen is to have a bunch of hopes, plans, dreams, and things, sitting in a backlog because they're too big to ship in the time that you've got.

Which brings us to the other key to agility ... ship things on a shorter schedule.

This re-trains your brain to chunk things down, flow value, chop dependencies down to size, learn, and, move on.

Best of all, if you miss the train, you catch the next train.

Categories: Architecture, Programming

Foq It Easy

Phil Trelford's Array - Wed, 04/24/2013 - 16:10

Foq is a .Net mocking library with first class support for F# and a cool sounding name. After 9 public releases, Foq is now mature and stable at version 0.9. What started as a 6 to 8 week project ended up more like a half-year project. At this point, I’d like to say a big thank you for the great feedback from the community, and specifically acknowledge the help and support of Mathias Brandewinder, Vasily Kirichenko, Niklas Bergius and Neil Danson. Yesterday the first Foq question appeared on Stack Overflow, thanks to Michael Newton.

F# makes unit testing easy with static method support in xUnit and NUnit, white space in method names and object expressions for mocking simple interfaces:

let [<Test>] ``at 15:00 the sun image should be expected`` () =
    let time = { new ITime with member mock.GetHour() = 15 }
    let calculator = ImageCalculator(time)
    let image = calculator.GetImageForTimeOfDay()
    Assert.AreEqual("sun.jpg",  image)

Foq provides a fluent interface for scaling up to larger interfaces like IList(T), where there are many members, but you may only be interested in specifying return values for a few of them:

let [<Test>] ``setup a method to always return true`` () =
    // Arrange
    let instance =
        Mock<IList>()
            .Setup(fun xs -> <@ xs.Contains(any()) @>).Returns(true)
            .Create()
    // Assert
    Assert.IsTrue(instance.Contains("Anything"))

Foq’s API is quite similar to Moq’s or FakeItEasy’s. Not only does it support LINQ expressions there’s F# Code Quotations too. I tend to think of Code Quotations like classic ASP code blocks <% .. %> for quoting code but with @’s <@ … @>. Code Quotations add a little syntactic noise but come with more expression types than LINQ expressions, so far example you can specify events as you would properties. If however you prefer your lambdas with less symbols then just open the Foq.Linq namespace instead.

Foq also provides some handy shortcuts like the Method method:

let [<Test>] ``setup a method to always return true`` () =
    // Arrange
    let instance =
        Mock<IList>.Method(fun xs -> <@ xs.Contains @>).Returns(true)
    // Assert
    Assert.IsTrue(instance.Contains("Anything"))

Note that even the arguments are omitted from the invocation of Contains for ease.

Where the Method method lets you set up a single method, the With method lets you setup multiple methods with a single method:

Mock<IList<char>>.With(fun xs ->
    <@ xs.Count --> 2 
       xs.Item(0) --> '0'
       xs.Item(1) --> '1'
       xs.Contains(any()) --> true
       xs.RemoveAt(2) ==> System.ArgumentOutOfRangeException()
    @>
)

Here a custom operator –-> is employed to signify returning a value from a member, and the ==> operator raising an exception. For setting up many members, letting go of the heavy fluent interface helps let the actual intent shine through.

Sometimes you just want an instance of an interface mocked which just follows the defaults, and like Moq you can easily specify Mock<T>.Of:

let [<Test>] ``order sends mail if unfilled`` () =
    // setup data
    let order = Order("TALISKER", 51)
    let mailer = Mock.Of<MailService>()
    order.SetMailer(mailer)
    // exercise
    order.Fill(Mock.Of<Warehouse>())
    // verify
    verify <@ mailer.Send(any()) @> once

The mock function lets F# type inference worry about the types for you:

let [<Test>] ``order sends mail if unfilled`` () =
    // setup data
    let order = Order("TALISKER", 51)
    let mailer = mock()
    order.SetMailer(mailer)
    // exercise
    order.Fill(mock())
    // verify
    verify <@ mailer.Send(any()) @> once

Foq support verification of calls as seen above, or you can specify expectations against members and even specific sequence of calls, as you would with the With method:

let xs = Mock<IList<int>>.ExpectSequence(fun xs ->
    <@
        xs.Clear()
        xs.[any()] --> any()
        xs.[any()] <- any()
        xs.Count --> any()
        xs.Contains(any()) --> any()
    @>)

Want to see some more examples, you got it:

All you need to play with Foq is to open an F# project in Visual Studio, Xamarin Studio or Tsunami and pull the Foq library from Nuget or simply reference Foq.fs in your project. Looking forward to more releases and more feedback :)

Categories: Programming

Learn the Dart language and libraries with these 11 short videos

Google Code Blog - Wed, 04/24/2013 - 14:57
Author PhotoBy Seth Ladd, Developer Advocate

Sometimes, you only have 5 minutes. Luckily, with the Dart Tips series of short video tutorials, that's all you need to start learning the new Dart language and libraries. In these videos, I show off code samples and examples across the various language features of Dart.

For example, here's a quick demo of the various types of constructors in Dart:



You can check out all the videos, or jump to the topic that most interests you:
  1. A simple Dart script
  2. Runtime modes
  3. Variables
  4. Strings, numbers, booleans, oh my!
  5. Collections in Dart
  6. Functions are fun, Pt 1
  7. Functions are fun, Pt 2
  8. Control flow statements
  9. Exceptions
  10. Classes: setters & getters
  11. Classes: Constructors
We hope you enjoy these videos about Dart. If you have questions or comments about the videos, or Dart, please join us on the Dart mailing list and on Google+. As we say on Dart Tips, stay sharp!


Seth Ladd is a Developer Advocate on Dart. He's a web engineer, book author, a conference organizer, and loves a game of badminton.

Posted by Scott Knaster, Editor
Categories: Programming

Unix: Checking for open sockets on nginx

Mark Needham - Wed, 04/24/2013 - 00:59

Tim and I were investigating a weird problem we were having with nginx where it was getting in a state where it had exceeded the number of open files allowed on the system and started rejecting requests.

We can find out the maximum number of open files that we’re allowed on a system with the following command:

$ ulimit -n
1024

Our hypothesis was that some socket connections were never being closed and therefore the number of open files was climbing slowly upwards until it exceeded the limit.

We wanted to check how many sockets nginx had open so to start with we needed to know the process IDs it was running under:

$ ps aux | grep nginx | grep -v grep
root      1089  0.0  0.7 105152  2736 ?        Ss   17:34   0:00 nginx: master process /usr/sbin/nginx
www-data 17474  0.0  0.6 105300  2296 ?        S    21:49   0:04 nginx: worker process
www-data 17475  0.0  0.7 105300  2856 ?        S    21:49   0:04 nginx: worker process
www-data 17476  0.0  0.7 105300  2792 ?        S    21:49   0:03 nginx: worker process
www-data 17477  0.0  0.7 105300  2668 ?        S    21:49   0:04 nginx: worker process

So the process IDs we’re interested in are 1089, 17474, 17475, 17476 and 17477.

We can check which file descriptors they have open with the following command:

$ sudo ls -alh /proc/{1089,17{474,475,476,477}}/fd
/proc/17476/fd:
total 0
dr-x------ 2 www-data www-data  0 Apr 23 23:40 .
...
l-wx------ 1 www-data www-data 64 Apr 23 23:40 6 -> /var/log/nginx/error.log
l-wx------ 1 www-data www-data 64 Apr 23 23:40 7 -> /var/www/thinkingingraphs/shared/log/nginx_access.log
l-wx------ 1 www-data www-data 64 Apr 23 23:40 8 -> /var/www/thinkingingraphs/shared/log/nginx_error.log
lrwx------ 1 www-data www-data 64 Apr 23 23:40 9 -> socket:[8910]
 
/proc/17477/fd:
total 0
...
lrwx------ 1 www-data www-data 64 Apr 23 23:40 56 -> socket:[52213]
lrwx------ 1 www-data www-data 64 Apr 23 23:40 57 -> anon_inode:[eventpoll]
l-wx------ 1 www-data www-data 64 Apr 23 23:40 6 -> /var/log/nginx/error.log
l-wx------ 1 www-data www-data 64 Apr 23 23:40 7 -> /var/www/thinkingingraphs/shared/log/nginx_access.log
l-wx------ 1 www-data www-data 64 Apr 23 23:40 8 -> /var/www/thinkingingraphs/shared/log/nginx_error.log
lrwx------ 1 www-data www-data 64 Apr 23 23:40 9 -> socket:[8910]

We can narrow that down to just show us how many sockets are open:

$ sudo ls -alh /proc/{1089,17{474,475,476,477}}/fd | grep socket  | wc -l
189

We could also use lsof although for some reason that returns a slightly different number:

$ sudo lsof -p 1089,17474,17475,17476,17477 | grep socket | wc -l
184

If we want to use brace expansion to do that it becomes a bit more tricky:

$ sudo lsof -p `echo {1089,174{74,75,76,77}} | sed 's/ /,/g'` | grep socket | wc -l
184

Annoyingly we couldn’t actually replicate the error but think that it’s been solved in nginx 1.2.0 (we were using 1.1.19) by this change:

Bugfix: a segmentation fault might occur in a worker process if the
       "try_files" directive was used; the bug had appeared in 1.1.19.
Categories: Programming

No downtime deploy with capistrano, Thin and nginx

Mark Needham - Wed, 04/24/2013 - 00:25

As I mentioned a couple of weeks ago I’ve been working on a tutorial about thinking through problems in graphs and since it’s a Sinatra application I thought thin would be a decent choice for web server.

In my initial setup I had the following nginx config file which was used to proxy requests on to thin:

/etc/nginx/sites-available/thinkingingraphs.conf

upstream thin {
  server 127.0.0.1:3000;
}
 
server {
  listen       80 default;
  server_name _;
  charset utf-8;
 
  rewrite  ^\/status(.*)$  $1 last;
 
  gzip  on;
  gzip_disable "MSIE [1-6]\.(?!.*SV1)";
  gzip_types       text/plain application/xml text/xml text/css application/x-javascript application/xml+rss text/javascript application/json;
 
  gzip_vary on;
 
  access_log  /var/www/thinkingingraphs/shared/log/nginx_access.log;
  error_log  /var/www/thinkingingraphs/shared/log/nginx_error.log;
 
  root   /var/www/thinkingingraphs/current/public;
 
  location / {
    proxy_pass http://thin;
  }
 
  error_page  404              /404.html;
  error_page   500 502 503 504  /500.html;
}

I had an upstart script which started the thin server…

/etc/init/thinkingingraphs.conf

script
  export RACK_ENV=production
  export RUBY=ruby
 
  cd /var/www/thinkingingraphs/current
  exec su -s /bin/sh vagrant -c '$RUBY -S bundle exec thin -p 3000 start >> /var/www/thinkingingraphs/current/log/production.log 2>&1'
end script

… and then I used the following capistrano script to stop and start the server whenever I was deploying a new version of the application:

config/deploy.rb

namespace :deploy do
  task(:start) {}
  task(:stop) {}
 
  desc "Restart Application"
  task :restart do
    sudo "stop thinkingingraphs || echo 0"
    sudo "start thinkingingraphs"
  end
end

The problem with this approach is that some requests receive a 502 response code while its restarting:

$ bundle exec cap deploy
$ while true; do curl -w %{http_code}:%{time_total} http://localhost/ -o /dev/null -s; printf "\n"; sleep 0.5; done
 
200:0.076
200:0.074
200:0.095
502:0.003
200:0.696

I wanted to try and make a no downtime deploy script and I came across a couple of posts which helped me work out how to do it.

The first step was to make sure that I had more than one thin instance running so that requests could be sent to one of the other ones while a restart was in progress.

I created the following config file:

/etc/thin/thinkingingraphs.yml

chdir: /var/www/thinkingingraphs/current
environment: production
address: 0.0.0.0
port: 3000
timeout: 30
log: log/thin.log
pid: tmp/pids/thin.pid
max_conns: 1024
max_persistent_conns: 100
require: []
wait: 30
servers: 3
daemonize: true
onebyone: true

One of the other properties that we need to set is ‘onebyone’ which means that when you restart thin it will take down the thin instances one at a time. This means one of the other two can handle incoming requests.

We’ve set the number of servers to 3 which will spin up 3 instances on ports 3000, 3001 and 3002.

I changed my upstart script to look like this:

/etc/init/thinkingingraphs.conf

script
  export RACK_ENV=production
  export RUBY=ruby
 
  cd /var/www/thinkingingraphs/current
  exec su -s /bin/sh vagrant -c '$RUBY -S bundle exec thin -C /etc/thin/thinkingingraphs.yml start >> /var/www/thinkingingraphs/current/log/production.log 2>&1'
end script

I also had to change the capistrano script to call ‘thin restart’ instead of stopping and starting the upstart script:

config/deploy.rb

namespace :deploy do
  task(:start) {}
  task(:stop) {}
 
  desc "Restart Application"
  task :restart do
    run "cd #{current_path} && bundle exec thin restart -C /etc/thin/thinkingingraphs.yml"
  end
end

Finally I had to make some changes to the nginx config file to send on requests to other thin instances if the first attempt failed (due to it being restarted) using the proxy_next_upstream method:

/etc/nginx/sites-available/thinkingingraphs.conf

upstream thin {
  server 127.0.0.1:3000 max_fails=1 fail_timeout=15s;
  server 127.0.0.1:3001 max_fails=1 fail_timeout=15s;
  server 127.0.0.1:3002 max_fails=1 fail_timeout=15s;
}
 
server {
  listen       80 default;
  server_name _;
  charset utf-8;
 
  rewrite  ^\/status(.*)$  $1 last;
 
  gzip  on;
  gzip_disable "MSIE [1-6]\.(?!.*SV1)";
  gzip_types       text/plain application/xml text/xml text/css application/x-javascript application/xml+rss text/javascript application/json;
 
  gzip_vary on;
 
  access_log  /var/www/thinkingingraphs/shared/log/nginx_access.log;
  error_log  /var/www/thinkingingraphs/shared/log/nginx_error.log;
 
  root   /var/www/thinkingingraphs/current/public;
 
  location / {
    proxy_pass http://thin;
    proxy_next_upstream error timeout http_502 http_503;
  }
 
  error_page  404              /404.html;
  error_page   500 502 503 504  /500.html;
}

We’ve also made a change to our upstream definition to proxy requests to one of the thin instances which will be running.

When I deploy the application now there is no downtime:

$ bundle exec cap deploy
$ while true; do curl -w %{http_code}:%{time_total} http://localhost/ -o /dev/null -s; printf "\n"; sleep 0.5; done
 
200:0.094
200:0.095
200:0.082
200:0.102
200:0.080
200:0.081

The only problem is that upstart now seems to have lost a handle on the thin processes and from what I can tell there isn’t a master process which upstart could get a handle on so I’m not sure how to wire this up.

Any ideas welcome!

Categories: Programming

The Thinker: Michael Lopp to Keynote Distill

Engine Yard Blog - Tue, 04/23/2013 - 18:00

While we as a community spend time thinking about how to write great code, minimize bugs, determine the right database schema, anticipate platform shifts and more, Michael Lopp is thinking about the mindset of the developers in our community.  Michael thinks deeply about the problems developers face in their everyday lives, how to help developers identify their true goals, what makes them happy and how to achieve happiness, how to lead and how to follow and much more.  Michael has spoken at every FunConf I have hosted since its inception and has consistently given our attendees a more authentic, meaningful way to think about our lives, what we do and the implications.  Michael is one of the great thinkers in our community and we are very pleased to have him be a keynote speaker at Distill.  Here's a little more information on Michael:

Michael Lopp is a director at Palantir Technologies, a Silicon Valley software company dedicated to radicalizing the way the world interacts with data.  Before joining Palantir, Lopp was part of the senior leadership team at Apple for nine years where he led essential parts of the Mac OS X engineering team, and subsequently managed the engineering team responsible for Apple's Online Store. Prior to Apple, he worked in engineering leadership at notable Silicon Valley companies such as Netscape, Symantecand Borland. Lopp is a noted author in Silicon Valley; his blog. “Rands In Repose,” and his books, Managing Humans and Being Geek, are part of a new management and engineering canon.

Distill is a conference to explore the development of inspired applications. Tickets go on sale this week.

Categories: Programming

Keep your CRUD off the Internet

Eric.Weblog() - Eric Sink - Tue, 04/23/2013 - 17:00

Whenever I explain database sync for mobile, somebody asks, "What kinds of apps would benefit from that?"

The correct answer is: "ALL apps".

Actually, that's an exaggeration. I see no reason for my alarm clock or calculator apps to sync data with a server. Let me rephrase:

Sync is the best architecture
for apps where a database and a server are involved
.

CRUD over REST is an Anti-Pattern

Many apps today keep the data only on the server and access it using REST. This approach has some big negative consequences:

  • Performance: Your app is less responsive.

    A round-trip from a mobile device to a server takes a LONG time. Under good network conditions, it could be a quarter of a second. That's enough time for the user to perceive a lag.

    Sometimes the round-trip will take longer, more like a full second, or more. That's an eternity, a one-star review in the App Store.

    Numerous studies show that when it comes to the responsiveness of a user interface, fractions of a second matter a LOT. And people don't necessarily complain about your app. They just stop using it.

    If you're going to make a round-trip to the server in response to a user action, your app is going to feel sluggish.

  • Reliability: Your app doesn't work offline.

    WiFi is great, but not everything is a coffee shop.

    Cellular data networks have dead spots, sometimes in surprising places. Even in areas with solid 3G or LTE coverage, the latency varies wildly.

    If your app is going to stop working when the wireless network is unavailable or unreliable, then your app is going to stop working a lot.

  • Complexity: Your app's code has to deal with networking issues everywhere.

    People wrap their REST calls in pretty wrappers to make them look like networking is not involved. But it is. A robust app needs to deal with all the things that can go wrong.

    If you're going to do all your basic database operations over a REST API, all your code is networking code.

The real question is "What kinds of apps would benefit from having user interaction dependent on a wireless network with inconsistent coverage and random latency spikes?"

Your app does lots of basic database operations which are often referred to as CRUD (Create, Read, Update, Delete). These operations do not want to happen over a network. They want to happen in a database which is local, on the mobile device.

And so you need sync. In the background (so the user doesn't have to wait on it). With automatic conflict resolution.

And that's hard.

Solutions that make sync easy

Do you like SQL? If so, I invite you to look at Zumero, our database sync platform based on SQLite, the relational database software preinstalled on over a billion mobile devices.

Do you prefer NoSQL? Then I'm sure our worthy competitors over at Couchbase would be happy for you to look at Couchbase Lite.

Either way, keep your CRUD off the Internet.

 

Elizabeth Edersheim on Management Lessons of a Lifelong Student

I’m always on the lookout for the best insight and action you can use for work and life.  I especially enjoy when I find somebody who is truly a thought leader, a giant in their space.

After all, I’m a big fan of helping everyone “stand on the shoulders of giants.”

Elizabeth is a giant (actually, more like a Titan) in the field of management.   She brings to the table more than 30 years of experience in the art and science of management.  She’s a former McKinsey partner, a holds a PhD from the Massachusetts Institute of Technology’s Sloan School of Management, and she is the author of McKinsey’s Marvin Bower, and The Definitive Drucker.

She knows her stuff.

So I asked her to share her stuff.

Elizabeth has written a powerful guest post for me on her best lessons learned in the art and science of management:

Management Lessons of a Lifelong Student, by Elizabeth Edersheim.

She reveals the secrets of the best managers and best leaders, and puts it right at your fingertips.  Every now and then you read something that changes your breadth or depth on a topic.   This is one of those posts.

It’s a wealth of insight and action.

Keep in mind that Elizabeth operates at multiple levels of management, so whether you are a line-leader or a CEO, Elizabeth has distilled some key insights you can immediate apply, or refine your thinking, or perhaps lead to a new “ah-ha” moment.

Enjoy, and may the best practices for management serve you well, whether you’re shaping your own business or the business around you.

Categories: Architecture, Programming

Episode 193: Apache Mahout

Recording Venue: Skype Guest: Grant Ingersoll Grant Ingersoll, founder of the Mahout project, talks with Robert about machine learning.   The conversation begins with an introduction to machine learning and the forces driving the adoption of this technique. Grant explains the three main use cases, similarity metrics, supervised versus unsupervised learning, and the use of large data [...]
Categories: Programming

A new kind of summer job: open source coding with Google Summer of Code

Google Code Blog - Mon, 04/22/2013 - 20:21
By Stephanie Taylor, Open Source team

Cross-posted from the Official Google Blog

If you’re a university student with CS chops looking to earn real-world experience this summer, consider writing code for a cool open source project with the Google Summer of Code program.


Over the past eight years more than 6,000 students have “graduated” from this global program, working with almost 400 different open source projects. Students who are accepted into the program will put the skills they have learned in university to good use by working on an actual software project over the summer. Students are paired with mentors to help address technical questions and concerns throughout the course of the project. With the knowledge and hands-on experience students gain during the summer they strengthen their future employment opportunities in fields related to their academic pursuits. Best of all, more source code is created and released for the use and benefit of all.

Interested students can submit proposals on the website starting now through Friday, May 3 at 12:00pm PDT. Get started by reviewing the ideas pages of the 177 open source projects in this year’s program, and decide which projects you’re interested in. Because Google Summer of Code has a limited number of spots for students, writing a great project proposal is essential to being selected to the program. Be sure to check out the Student Manual for advice.

For ongoing information throughout the application period and beyond, see the Google Open Source blog, join our Summer of Code mailing lists or join us on Internet relay chat at #gsoc on Freenode.

Good luck to all the open source coders out there, and remember to submit your proposals early—you only have until May 3 to apply!


Written by Stephanie Taylor, Open Source team

Posted by Scott Knaster, Editor
Categories: Programming

Privacy Is Dead, Time To Prepare For The Future

Making the Complex Simple - John Sonmez - Mon, 04/22/2013 - 13:00

Personal privacy is over.

The world knows more about you than you do and soon it will know even more.

We can keep fighting the battle to secure our privacy or we can learn how we need to live in the coming age where all of our actions are potentially in public view.

Before you get worried that I am going all political on you, don’t worry, I’m not condoning the invasion of personal privacy or the eradication of it, but at the same time I’m not supporting it either.

I’m simply looking at the patterns that are emerging as the technology of our society increases, making this transformation inevitable.

I’m actually staunchly opposed to expressing my political thoughts publicly from any forum, even from my own blog, and I often recommend others do the same, mostly because of this transformation that is taking place.  I’ll talk more about that a little later on, but first let me tell you why privacy is indeed dead or at the very least dying.

It’s more than Google Glass

Google Glass itself may not change the concept of personal privacy, but wearable computing is certainly the future.

Google Glass helps end privacy

Google Glass

What Google Glass does do, is sets up the stage for wearable computing to go mainstream.

I hear many people complaining about Google Glass saying that they would never wear the technology in public, because they would look like a geek or a dork.

But, this is exactly what makes innovations like Glass so important.  The first thing that changes in society is perceptions.

Google Glass may not have the momentum to bash through the barrier of the “dorkiness” of wearing a computer on your head, but it will make a dent in that wall and over time that wall will come crashing down.

When everyone is wearing cameras it will be impossible to control them

We’ve already been through this before.

We’ve been through it at least two times that I can think of.

First, there was the USB drive.

Corporate privacy is easily averted

All of a sudden we could plug a small device into just about any computer and pull data off of it or put data on it.

Computers without disk drives or network access were no longer inaccessible.

The data on them was no longer securable.

Sure, many companies tried to make it so PCs in their buildings were secured, but most companies realized that they would just have to deal with this new reality.  You had to realized that if a person had access to data on a computer, they could take that data home if they wanted to.

The same type of thing happened when all cell phones started getting quality cameras on them.

Secure facilities tried to prevent people from taking pictures by disallowing employees to bring in cell phones that had cameras, but that didn’t last long.

It became apparent pretty quickly that it would be just about impossible to prevent human beings who now were part human, part iOS or Android, from bringing their cell phone into a building.  And, even if you could, someone who really wanted to take a picture would find another way.  The technology that brought tiny cameras to the backs of every cell phone also pioneered the mass creation of tiny cameras that could easily be hidden anywhere.

Already governments and family matters have been thrust into public view

We currently live in a society where it is just about impossible for public figures or government to do an action in public and try to keep it secret.

Think about all the times in the last few years when a police officer stepped over the line or a government tried to quell a public protest by using violence or excessive force.

Each and every time, not only was there photographs and videos of the incidents, but the information spread to the entire world almost instantly through mediums like Twitter and Facebook.

The fact of the matter is, it is almost impossible to hide or cover up anything that occurs in public view.

Consider the amazing and tragic events that recently occurred at the Boston Marathon.  In a huge crown of people, it only took about a day to identify two suspects out of the thousands of people in the crowd using images and video provided by the public.

This event was significant, because it showed that even when people aren’t trying to capture data about you in public, they are capturing data.

You don’t have to worry about the government tracking you. The government doesn’t have the power or capability to do it.  Instead, you are tracking yourself, you are tracking the government.

Oh, and if you still think family matters are private, you probably haven’t heard of this thing called Facebook.

Don’t think you can get a divorce, or even get into a heated argument with your teenager and not have it become public knowledge on Facebook.

The problem is that there are too many leaks now to private family matters and it is too easy for those leaks to spread the information.

Worse yet, those naked pictures you took of yourself, or unfortunately your less worldly-wise teenager took of themselves, that you thought no one would see… all I can say is “good luck with that.”

The reality is the trend against privacy will not reverse

Not only are you willingly providing your own private information, but companies are actively mining it.

From tracking cookies, to search histories, to friends graphs, advertising link clicks, foursquare check-ins, what you post on Twitter and more, everything is being recorded and tracked.

Each piece of information by itself is not very valuable, but when you combine all this information together, a very real and precise silhouette begins to emerge from that frosted glass window you are hiding behind.

And, like I said before, even if Google Glass is not successful, it will not be long before every human walking around in public and in private wearing a camera that not only you can’t control, but likely you won’t even be able to see.

Not only will we all eventually be wearing cameras, but the data from those cameras will be instantly uploaded to the cloud, where it will become a permanent record.

There are several unstoppable factors that will continue to improve over time, which will fuel this trend:

  • Miniaturization of technology
  • Increase in efficiency of batteries
  • Increase in processing power
  • Prevalence of social networks

As technology gets smaller and is able to be more mobile, and the ability to crunch big data and distribute data through social networks increases, privacy will greatly diminish.

The reality is we are transitioning from a society where private is default and public is a choice, to one where public is default and making things private requires considerable effort and in some cases is impossible.

How to live in this new “public by default” world

Rather than uselessly spending our energies trying to fight the trend of the world, it is better to spend those energies tempering our behavior to fit the world we are living in and will soon be born into.

It starts with getting in the habit of assuming everything we put on the internet will not only be made public, but will likely be a permanent record that is easily searchable.

Many times throughout the day I watch my Twitter or Facebook feed fly by and I cringe at all the people who are posting their own private opinions that should not be made public.

I’m not saying you can’t have an opinion, but I am saying that you have to be wary that the opinions you express will become visible by more than just the people you share them with and will become a permanent record which you cannot be alienated from.

Are you a republican?  Are you a democrat?  Perhaps you are a libertarian or a communist?  Do you support gun control?  Are you pro-choice?

Good!  Stand for something, but there isn’t much of a need to say it out loud.  Let your actions stand for you, they are more powerful than words anyway.

The problem with tweeting about your political views is that they may change.  You might be leaning one political direction today, but 5 years from now, you have changed your mind completely.  But, guess what won’t ever change?  That’s right, the permanent record of what you said on the internet.  If you ever decide to run for office or even go on a date, those words you said may come back to haunt you.

And even if your mind doesn’t change, do you really want to decisively set half of the world population directly against you?  I can’t image many scenarios where this is a good idea.  Even in politics, bipartisan political figures are usually much more successful.  (Ok, I made that up, but I think it’s true.)

I’m getting a little bit away from the point, but what I am trying to say is that you need to strongly consider things before you put them on the internet, because what you say online is public and permanent.

It goes beyond just the internet of course, even though it may start there.

We have to prepare ourselves for a world in which the moment we step out our door our actions are recorded.

In this kind of world, we have to be careful not to try to hide things that are likely to be destructive to us if revealed, because we know that what is hidden is likely to be revealed.

It means being more upfront and honest in our dealings and being careful to manage our image to make the focus be on the images and presentation that we desire while diminishing the focus on the image we do not desire.

When you don’t have control over the information itself, you have to instead focus on controlling how that information is presented.

It all comes down to living in such a way that clearly reflects the values you are trying to represent and knowing that perception is 100 times more important than reality.

There is no time like the present

If you are still living like you can hide your tracks and like your track record doesn’t matter, you’d better stop now, before it’s too late!

The answer isn’t to try to find better hiding places for your secrets and to try to wall yourself away from the world, but instead of accept the reality of the situation and find the best way to adapt to it.

“Man is not what he thinks he is, he is what he hides.”
― André Malraux

If you like this post don’t forget to Follow @jsonmez or subscribe to my RSS feed.

!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');

The post Privacy Is Dead, Time To Prepare For The Future appeared first on Making the Complex Simple.

Categories: Programming

Windows AzureConf this Tuesday

ScottGu's Blog - Scott Guthrie - Mon, 04/22/2013 - 05:59

This Tuesday, April 23, we’ll be hosting Windows AzureConf – a free online event for and by the Windows Azure community.  It will be streamed online from 9:00 AM - 5:00 PM PST via Channel 9, and you can watch it all for free.

I’ll be kicking off the event with a Windows Azure keynote in the morning (a great way to learn more about Windows Azure if you haven’t used it yet!). Following my talk the rest of the day will be full of excellent presentations from members of the Windows Azure community.  You can ask questions from them live and I think you’ll find the day an excellent way to learn more about Windows Azure – as well as hear directly from developers building solutions on it today.

Last year’s Windows AzureConf was a great success, and brought some awesome community members together to deliver some great content around Windows Azure. All of the sessions are available for on-demand viewing on the Windows AzureConf 2012 event page on Channel 9. Sessions from Windows AzureConf 2012 are still available for viewing online.

For more information including a schedule, speaker list or to register visit the Windows AzureConf website.

Hope to see you there!

Scott

P.S. We will also make the presentations available for download after the event in case you miss them.

clip_image001

Categories: Architecture, Programming

Developing a SOA-based Integration Layer Framework: Features

Xebia Blog - Sun, 04/21/2013 - 22:12

A few years ago I was asked by one of our customers to help them make better use of their integration layer. Ever since then me and my team have been working on a framework in support of that. This is the fourth in a series of blogs on the development of our framework, and discusses the features it provides. The one that was announced last time, about building blocks, is momentarily postponed.

So far I’ve discussed the goals & challenges surrounding the development activities, but I’d like to focus more on the framework itself now, and what it brings to those that are using it.

As soon as a new party (be it service consumer or service provider) connects to our framework, it can profit directly from the wealth of functionality we deliver out-of-the-box. These ‘generic features’ are exactly what one would expect from a (logical) ESB, and are partly based on the Expanded Enterprise Service Bus Pattern.

esb

As our project was scrum driven, features were developed only when they were needed. Sometimes, during the design & build phase, we discovered a better way of doing things, and sometimes the problem a feature was supposed to address was solved in a completely different way, outside of our scope. But in the end we managed to implement about 20 features, which can roughly be divided into five types: routing, robustness, security, transformation and data storage.

Routing

One of our main objectives is to get messages from A to B without A having to know where B is currently residing. To do this, we make extensive use of the WS-Addressing standard. One of the components in our framework, the routing service, uses the information in the message headers to decide what the next hop will be (hop in this case being another framework component). Most of the times a message is delivered to the backside as soon as it enters the integration layer, something we call simple routing.

However, as soon as some special activity needs to be performed (like data model transformation for example), the message is detoured to one of the framework components not connected to the outside world. We classify those as intermediate services, and they are agnostic of nature – which means they have no clue about the context in which they are called. The necessary information for the message to continue its path is part of the addressing headers as well, making this advanced routing.

A special kind of advanced routing is distribution, which makes it possible to send one message to several subscribers at the same time, using WS-Notification in its most basic form. Finally prioritized routing is a feature which makes sure that a message gets ahead of the rest so to speak – very useful when dealing with a customer waiting for service at a counter while there’s also a bulk load being processed.

Robustness

Of course it’s of eminent importance that nothing goes wrong when delivering the message, or that when it does we can at least deal with the situation (exception management). First thing that happens when we receive a message is that we check to see if it complies with the industry & design standards we enforce (technical validation). Sometimes the consumer/publisher doesn’t want to wait for the (functional) answer but still wants to be informed if his message was technically valid, in which case we send him a response stating just that (technical acceptance).

Two of our features deal with peak load: throttling makes sure the integration layer only takes in what it can handle, while buffering guarantees we don’t overwhelm the applications we connect to. Similar to the second one is postponed delivery which is used when we know beforehand the backside is not available.

But by far the most important of these types of feature is guaranteed delivery. We played around with a bidirectional variant (using WS-RM) but finally settled on an unidirectional implementation, meaning that before we send the message we first store it, and if we receive an HTTP error code we send it again.

Last (and actually least, as it’s hardly used) it’s also possible to have syntactic validation of outgoing messages. But as we like to follow Postel’s law (also known as the robustness principle) we feel it’s the consumer’s responsibility to make sure the message was valid to begin with (you can imagine this took some selling from our part). The only exception is when the payload is altered by the framework.

Security

Every application that wants to connect to the integration layer needs to make itself known using the WS-Security UsernameToken (authentication). For most of the services we expose that’s enough, in a few rare cases such an application has to have explicit permission (authorization).

Not used internally (only when we receive requests from certain external customers) is integrity, where we demand that certain parts of the message headers and payload are signed.

Transformation

Given the fact that not all applications connected to the framework ‘speak the same language’ (as mentioned in the previous blog), there’s an evident need for data model transformation – one of our most used features. A lot less popular is the split feature, which makes it possible to divide a big message into smaller parts.

Data storage

The last few features play a more supportive role to the ones already mentioned. We provide logging to be used during testing & bug-finding sessions and persistence when there’s a need to store the complete message. The latter is frequently used in combination with resending (which is necessary for the guaranteed delivery feature described above), but also in case of auditing requirements.

Conclusion

Most of these features have been around since the first version of our framework, and have proven their generic qualities over time. In a few cases we had to make some alterations and even now there are one or two features which we might implement differently in the future. There’s also a list of additional features but it’s rather short, which I take as a sign that what we have here is pretty complete.

That’s number four; next time I’ll talk a bit about the more specific deliverables we provide our project teams.

Wearable Computing

I was watching a video on Google Glass with Robert Scoble, and I couldn’t help but wonder about all the possibilities that technology can bring to the table.

Wearable computing bridges the gap between the real world and the things we see in Sci-Fi movies.

Of course, when we overlay information on our world, the key will be turning information into insight and action.  All change isn’t progress, and the market will flush out things faster than ever before.  And, to the victor go the spoils.

In the video, you can see how the Google Glass does a few basic things so far:

  1. Take a picture
  2. Record a video
  3. Get directions to ...
  4. Send a message to ...
  5. Make a call to ...

The big limit in what it’s capable of, so far, seems to be the batter power.  And of course, a key concern was security.  It’s another reminder how in the software space, security and performance always play a role, even if they are behind the scenes.  In fact, that’s the irony of software security and performance, they are at their best when you don’t notice them.

Security and performance are often unsung heroes.

The big take away for me is that the game is on warp speed now.  By game, I mean, the business of software.  You can go from idea to market pretty fast.   So the big bottlenecks range from the right ideas, to the right people, to the right strategy, to the right execution.

But more importantly, the reminder is this:

Companies with smart people, data-driven insights, a culture of innovation, great software processes, customer focus, and reach around the world, can change the world -- at a faster pace than ever before.

Who knows what we’ll be wearing next?

Categories: Architecture, Programming

ASP.NET Web API: CORS support and Attribute Based Routing Improvements

ScottGu's Blog - Scott Guthrie - Fri, 04/19/2013 - 19:37

We’ve seen a huge adoption of ASP.NET Web API since its initial release.  In February we shipped the ASP.NET and Web Tools 2012.2 Update – which added a number of additional enhancements to both Web API and the other components of ASP.NET. 

The ASP.NET Team has been hard at work on developing the next set of features (lots of cool stuff coming).  One of the great things about this work has been how the team has used the open source development process – which we announced we were adopting last spring - to collaborate even more closely with the community to both validate the features early, as well as enable developers in the community to directly contribute to the development of them.

Below are some updates on two of the great features coming to ASP.NET Web API – which were developed and contributed by ASP.NET MVP Brock Allen and Tim McCall (of attributerouting.net fame):

CORS support for ASP.NET Web API

Cross-origin resource sharing (CORS) is a W3C standard that allows web pages to make AJAX requests to a different domain. This standard relaxes the same-origin policy implemented in web browsers that restricts calls to the domain of the resource that makes the call. The CORS specification defines how the browser and server interact to make cross-origin calls.

The following image shows the ASP.NET Web API Test Tool (running on http://xyz123.azurewebsites.net/) making a cross domain call to the Contoso domain. When you click Send, a cross-origin request is made. Because the Contoso site is not configured to support CORS, an error dialog is displayed.

clip_image002[4]

The CORS error appears on the Console tab of the IE F12 tools.

clip_image004[4]

For security reasons, the web browser doesn’t allow calls from the azurewebsites domain to the Contoso domain. With the new ASP.NET Web API CORS framework, Contoso.com can be configured to send the correct CORS headers so the browser will accept cross-origin calls.

clip_image005[4]

MVP Brock Allen contributed his CORS source to the ASP.NET Webstack repository. Brock worked with Yao Huang Lin (a developer on the ASP.NET team), to refine and iterate the design and then to get it pulled into the Webstack repository. Brock Allen, Dan Roth, and Yao discuss Brock’s CORS contribution in this Channel 9 video.

The CORS support for ASP.NET Web API page shows how to get started with this new feature.

Attribute-Based Routing in ASP.NET Web API

We recently published in the ASP.NET Web API roadmap our intention to support attribute- based routing in ASP.NET Web API. Route attributes bring the URL definition closer to the code that runs for that particular URL, making it easier to understand which URL must be called for a particular block of code and simplifying many common routing scenarios.

For example, let’s say you want to define a Web API that has the standard set of HTTP actions (GET, POST, PUT, DELETE, and so on) but you also want to have an additional custom action, such as Approve. Instead of adding another route to the global route table for the Approve action, you can instead just attribute the action directly:

    public class OrdersController : ApiController

    {

        public IEnumerable<Order> GetOrders() {…}

        public Order GetOrder(int id) {…}

        public Order Post(Order order) {…}

        [HttpPost("orders/{id}/approve")]

        public Order Approve(int id) {…}

    }

An extended route template syntax makes it simple to specify default values and constraints for route values. For example, you can now easily create two actions that are called based on parameter type. In the following People controller, the id parameter of the GetByID action takes only int values. The GetByName action method contains a default name of “Nick”.

    public class PeopleController : ApiController

    {

        [HttpGet("{name=Nick}")]

        public string GetByName(string name) {…}

 

        [HttpGet("{id:int}")]

        public string GetById(int id) {…}

    }

You can also define common route prefixes for your web APIs. For example, you can use route prefixes to set up a resource hierarchy:

    [RoutePrefix("movies")]

    [RoutePrefix("actors/{actorId}/movies")]

    [RoutePrefix("directors/{directorId}/movies")]

    public class MoviesController : ApiController

    {

        public IEnumerable<Movie> GetMovies() {…}

        public IEnumerable<Movie> GetMoviesByActor(int actorId) {…}

        public IEnumerable<Movie> GetMoviesByDirector(int directorId) {…}

    }

Or, you can use route prefixes to handle multiple versions of your web API:

    [RoutePrefix("api/v1/customers")]

    public class CustomersV1Controller : ApiController {…}

   

    [RoutePrefix("api/v2/customers")]

    public class CustomersV2Controller : ApiController {…}

Similar to the new CORS support in ASP.NET Web API, the new support for attribute-based routing is largely a contribution from the community. We are working closely with Tim McCall of attributerouting.net fame to bring many of the features of his AttributeRouting project directly into ASP.NET Web API.

It’s really exciting to see how these collaborations across the ASP.NET Team and the community are helping to move the ASP.NET platform forward!

Hope this helps,

Scott

P.S. In addition to blogging, I use Twitter to-do quick posts and share links. My Twitter handle is: @scottgu

Categories: Architecture, Programming

April 19, 2013: This Week at Engine Yard

Engine Yard Blog - Fri, 04/19/2013 - 19:04

Our hearts and thoughts are with Boston, Waco, and Chicago.

--Tasha Drew, Product Manager

Engineering Updates

Rails 4.0 beta1 (rails-4.0.0.beta1) is now in GA on our platform!

PHP is available in Early Access on Engine Yard Cloud! Learn all about using PHP with Engine Yard Cloud.

Do you PagerDuty? We do! We find it so useful and critical to maintaining a robust on-call system that we extended it to all of our Premium Support customers. This week our Operations Manager, Jamie Bleichner, announced an even deeper integration for Engine Yard’s Premium Support offering, now offering ZenDesk, NewRelic, Pingdom, Splunk, Nagios, and many other integrations out-of-the-box.

Social Calendar (Come say hi!)

Tuesday, April 23rd: Dublin, Ireland: the Mobile App Development Ireland meetup group is meeting to have an iOS development overview class.

Wednesday, April 24th - Friday, April 26th: Chef Conf, San Francisco! Engine Yard is sponsoring and a bunch of us are attending! Hope to see you there.

Thursday, April 25th: Dublin, Ireland: Node.js Dublin will be meeting with two speakers, Domnic Tarr covering “Streams in Node.js,” and Richard Roger reporting on “The anatomy of an app.”

Thursday, April 25th PDX Coder Dojo: K - 12 students and their parents can play, explore, and learn about coding and building software!

Articles of Interest

An in-depth article all about php on Engine Yard Cloud, by Ireland’s product manager extraordinaire Noah Slater!

Treehouse taught us how to work with iOS core and open source frameworks.

Categories: Programming

Tell Your Story and Build Your Brand

No, this isn't about "Once upon a time."  There are ways to know and share yourself with skill.  You can combine stories and branding to reveal the truths that help you stand out in the marketplace or workplace, and play to your competitive edge.

But the challenge is this -- unless you're a skilled marketer, how do you reveal the power of your brand in a more compelling way?

I'm not a marketer, and I don't play one on T.V., so I have to work at it.  The way I work at it, is I pay attention to the people that are outstanding at what they do.

So what do the people that are outstanding at this do? 

They focus on values.  Finding shared values is the key to building brands and building stronger relationships in everything you do ... in work, and in life.  Brand building is largely about creating clarity around the values the brand stands for.

A simple way is to start by just figuring out three attributes that you want your brand to be about.  For example:

  1. Simplicity
  2. Excellence
  3. Freedom

It needs to be believable.  You need to believe it, in your heart of hearts and soul of souls. 

Related to that, you need to know who your brand is for.  What are the values they share?  What are the boundaries of those values, and at what point, do you have polar opposites or create conflict?

Find the intersection.

That’s where the magic happens.

If you want to be relevant, you need to find the intersection of the values. 

Values are the ultimate lightening rod.

Categories: Architecture, Programming

Puppet: Installing Oracle Java – oracle-license-v1-1 license could not be presented

Mark Needham - Fri, 04/19/2013 - 00:36

In order to run the neo4j server on my Ubuntu 12.04 Vagrant VM I needed to install the Oracle/Sun JDK which proved to be more difficult than I’d expected.

I initially tried to install it via the OAB-Java script but was running into some dependency problems and eventually came across a post which specified a PPA that had an installer I could use.

I wrote a little puppet Java module to wrap the commands in:

class java($version) {
  package { "python-software-properties": }
 
  exec { "add-apt-repository-oracle":
    command => "/usr/bin/add-apt-repository -y ppa:webupd8team/java",
    notify => Exec["apt_update"]
  }
 
  package { 'oracle-java7-installer':
    ensure => "${version}",
    require => [Exec['add-apt-repository-oracle']],
  }
}

I then included this in my default node definition:

node default {
  class { 'java': version => '7u21-0~webupd8~0', }
}

Unfortunately when I ran that I ended up with the following error:

err: /Stage[main]/Java/Package[oracle-java7-installer]/ensure: change from purged to present failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install oracle-java7-installer' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
  java-common
Suggested packages:
...
Unpacking oracle-java7-installer (from .../oracle-java7-installer_7u21-0~webupd8~0_all.deb) ...
 
oracle-license-v1-1 license could not be presented
try 'dpkg-reconfigure debconf' to select a frontend other than noninteractive
 
dpkg: error processing /var/cache/apt/archives/oracle-java7-installer_7u21-0~webupd8~0_all.deb (--unpack):
 subprocess new pre-installation script returned error exit status 2
Processing triggers for man-db ...
Errors were encountered while processing:
 /var/cache/apt/archives/oracle-java7-installer_7u21-0~webupd8~0_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

I came across this post on Ask Ubuntu which explained a neat trick for getting around it by making it look like we’ve agreed to the licence. This is done by passing options to debconf-set-selections.

For a real server I guess you’d want some step where a person accepts the licence but since this is just for my hacking it seems to make sense.

My new Java manifest looks like this:

class java($version) {
  package { "python-software-properties": }
 
  exec { "add-apt-repository-oracle":
    command => "/usr/bin/add-apt-repository -y ppa:webupd8team/java",
    notify => Exec["apt_update"]
  }
 
  exec {
    'set-licence-selected':
      command => '/bin/echo debconf shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections';
 
    'set-licence-seen':
      command => '/bin/echo debconf shared/accepted-oracle-license-v1-1 seen true | /usr/bin/debconf-set-selections';
  }
 
  package { 'oracle-java7-installer':
    ensure => "${version}",
    require => [Exec['add-apt-repository-oracle'], Exec['set-licence-selected'], Exec['set-licence-seen']],
  }
}
Categories: Programming

dpkg/apt-cache: Useful commands

Mark Needham - Thu, 04/18/2013 - 22:54

As I’ve mentioned in a couple of previous posts I’ve been playing around with creating a Vagrant VM that I can use for my neo4j hacking which has involved a lot of messing around with installing apt packages.

There are loads of different ways of working out what’s going on when packages aren’t installing as you’d expect so I thought it’d be good to document the ones I’ve been using so I can find them more easily next time.

Finding reverse dependencies

A couple of times I found myself wondering how a certain package had ended up on the VM because I hadn’t specified that it should be installed so I wanted to know who had!

I wanted to find out the reverse dependency for the package. e.g. to find out who depended on make which we can find out with the following command:

$ apt-cache rdepends make
make
Reverse Depends:
...
  build-essential
  make:i386
  libc6-dev:i386
  open-vm-dkms
  mythbuntu-desktop
  broadcom-sta-source
...

The nice thing about ‘rdepends’ is that it will tell us reverse dependencies even for a package that we haven’t installed. This was helpful here as I had forgotten to install ‘build-essential’ and this made it obvious.

Finding which version of a package is installed

I added one of the Brightbox repositories to get a more recent Ruby version and noticed that something weird was going on with the version of ‘nginx-common’ that puppet was trying to install.

It seemed like one one my dependencies was trying to pull in the ‘latest’ version of ‘nginx-common’ which I’d expected to be ’1.1.19-1ubuntu0.1′.

By passing the ‘policy’ flag to apt-cache I was able to see that there was a recent version available via Brightbox:

$ apt-cache policy nginx-common
nginx-common:
  Installed: 1.1.19-1ubuntu0.1
  Candidate: 1:1.2.6-1~43~precise1
  Version table:
     1:1.2.6-1~43~precise1 0
        500 http://ppa.launchpad.net/brightbox/ruby-ng/ubuntu/ precise/main amd64 Packages
 *** 1.1.19-1ubuntu0.1 0
        500 http://us.archive.ubuntu.com/ubuntu/ precise-updates/universe amd64 Packages
        100 /var/lib/dpkg/status
     1.1.19-1 0
        500 http://us.archive.ubuntu.com/ubuntu/ precise/universe amd64 Packages
Finding which versions of a package are available

Another flag that we can pass to apt-cache is ‘madison’ which shows us the available versions for a package but doesn’t indicate which version is installed:

$ apt-cache madison nginx-common
nginx-common | 1:1.2.6-1~43~precise1 | http://ppa.launchpad.net/brightbox/ruby-ng/ubuntu/ precise/main amd64 Packages
nginx-common | 1.1.19-1ubuntu0.1 | http://us.archive.ubuntu.com/ubuntu/ precise-updates/universe amd64 Packages
nginx-common |   1.1.19-1 | http://us.archive.ubuntu.com/ubuntu/ precise/universe amd64 Packages
     nginx |   1.1.19-1 | http://us.archive.ubuntu.com/ubuntu/ precise/universe Sources
     nginx | 1.1.19-1ubuntu0.1 | http://us.archive.ubuntu.com/ubuntu/ precise-updates/universe Sources
     nginx | 1:1.2.6-1~43~precise1 | http://ppa.launchpad.net/brightbox/ruby-ng/ubuntu/ precise/main Sources
Finding which package a file belongs to

At some stage I wanted to check which exact package was installing nginx which I was able to do with the following command:

$ dpkg -S `which nginx`
nginx-extras: /usr/sbin/nginx

I had installed ‘nginx-common’ which I learn depends on ‘nginx-extras’ by using our ‘rdepends’ command:

$ apt-cache rdepends nginx-extras
nginx-extras
Reverse Depends:
  nginx-naxsi:i386
...
  nginx-common
Finding the dependencies of a package

I wanted to check the dependencies of the ‘ruby1.9.1′ package to see whether or not I needed to explicitly install ‘libruby1.9.1′ or if that would be taken care of.

Passing the ‘-s’ flag to dpkg let me check this:

$ dpkg -s ruby1.9.1
Package: ruby1.9.1
Status: install ok installed
Architecture: amd64
Version: 1:1.9.3.327-1bbox2~precise1
Replaces: irb1.9.1, rdoc1.9.1, rubygems1.9.1
Provides: irb1.9.1, rdoc1.9.1, ruby-interpreter, rubygems1.9.1
Depends: libruby1.9.1 (= 1:1.9.3.327-1bbox2~precise1), libc6 (>= 2.2.5)
Suggests: ruby1.9.1-examples, ri1.9.1, graphviz, ruby1.9.1-dev, ruby-switch
Conflicts: irb1.9.1 (<< 1.9.1.378-2~), rdoc1.9.1 (<< 1.9.1.378-2~), ri (<= 4.5), ri1.9.1 (<< 1.9.2.180-3~), ruby (<= 4.5), rubygems1.9.1
...

These are the ones that I’ve found useful so far. I’d love to here other people’s favourites though as I’m undoubtably missing some.

Categories: Programming