Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Feed aggregator

SPaMCAST 346 ‚Äď Jon Quigley, Configuration Management

Software Process and Measurement Cast - Sun, 06/14/2015 - 22:00

The Software Process and Measurement Cast features our interview with Jon Quigley.  We discussed configuration management and his new book Configuration Management: Theory, Practice, and Application. Jon co-authored the book with Kim Robertson. Configuration management, the management and control of project deliverable, is one of the most critical practices anyone building a product, writing a piece of code or working on a project with more than one person involved must learn or face the consequences!

Jon’s Bio:

Jon M. Quigley PMP CTFL is a principal and founding member of Value Transformation, a product development training and cost improvement organization established in 2009, as well as being Electrical / Electronic Process Manager at Volvo Trucks North America.  Jon has an Engineering Degree from the University of North Carolina at Charlotte, and two Master Degrees from City University of Seattle.  Jon has nearly twenty five years of product development experience, ranging from embedded hardware and software through verification and project management.

Jon has written or contributed to a huge number of books, presentations and articles including:

‚ÄúConfiguration Management: Theory, Practice, and Application‚ÄĚ ISBN 978-148222935 May 1, 2015 which was the focus of this interview.

Jon is collaborating on another Project Management book (Q1 2016) through The Society of Automotive Engineers (SAE).

Contact Jon at:

jon.quigley@valuetransform.com

http://www.valuetransform.com/

https://www.linkedin.com/company/value-transformation-llc

https://www.linkedin.com/in/jonmquigley

Call to action!

Reviews of the Podcast help to attract new listeners.  Can you write a review of the Software Process and Measurement Cast and post it on the podcatcher of your choice?  Whether you listen on ITunes or any other podcatcher, a review will help to grow the podcast!  Thank you in advance!

Re-Read Saturday News

The Re-Read Saturday focus on Eliyahu M. Goldratt and Jeff Cox’s The Goal: A Process of Ongoing Improvement began on February 21nd. The Goal has been hugely influential because it introduced the Theory of Constraints, which is central to lean thinking. The book is written as a business novel. Visit the Software Process and Measurement Blog and catch up on the re-read.

Note: If you don’t have a copy of the book, buy one.  If you use the link below it will support the Software Process and Measurement blog and podcast.

Dead Tree Version or Kindle Version 

Next . . . The Mythical Man-Month Get a copy now and start reading! We will start in two weeks!

Upcoming Events

Software Quality and Test Management 

September 13 ‚Äď 18, 2015

San Diego, California

http://qualitymanagementconference.com/

I will be speaking on the impact of cognitive biases on teams!  Let me know if you are attending!

More on other great conferences next week.

Next SPaMCast

The next Software Process and Measurement Cast will feature our essay on Managing Agile Projects and Teams (7/15 -7/20/13 and last week updates ‚Äď Meg). Teams make decisions daily that affect the direction of the sprint and project. The faster these decisions are made the higher the team‚Äôs velocity or productivity, and having a solid understanding of the real goals of the project helps the team make decisions more effectively.

We will also have columns from Kim Pries with his Software Sensei column and Gene Hughson with an entry from his Form Follows Function column.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques¬†co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: ‚ÄúThis book will prove that software projects should not be a tedious process, neither for you or your team.‚ÄĚ Support SPaMCAST by buying the book¬†here.

Available in English and Chinese.

Categories: Process Management

SPaMCAST 346 ‚Äď Jon Quigley, Configuration Management

 www.spamcast.net

http://www.spamcast.net

Listen Now

Subscribe on iTunes

The Software Process and Measurement Cast features our interview with Jon Quigley.  We discussed configuration management and his new book Configuration Management: Theory, Practice, and Application. Jon co-authored the book with Kim Robertson. Configuration management, the management and control of project deliverable, is one of the most critical practices anyone building a product, writing a piece of code or working on a project with more than one person involved must learn or face the consequences!

Jon’s Bio:

Jon M. Quigley PMP CTFL is a principal and founding member of Value Transformation, a product development training and cost improvement organization established in 2009, as well as being Electrical / Electronic Process Manager at Volvo Trucks North America.  Jon has an Engineering Degree from the University of North Carolina at Charlotte, and two Master Degrees from City University of Seattle.  Jon has nearly twenty five years of product development experience, ranging from embedded hardware and software through verification and project management.

Jon has written or contributed to a huge number of books, presentations and articles including:

‚ÄúConfiguration Management: Theory, Practice, and Application‚ÄĚ ISBN 978-148222935 May 1, 2015 which was the focus of this interview.

Jon is collaborating on another Project Management book (Q1 2016) through The Society of Automotive Engineers (SAE).

Contact Jon at:

jon.quigley@valuetransform.com

http://www.valuetransform.com/

https://www.linkedin.com/company/value-transformation-llc

https://www.linkedin.com/in/jonmquigley

Call to action!

Reviews of the Podcast help to attract new listeners.  Can you write a review of the Software Process and Measurement Cast and post it on the podcatcher of your choice?  Whether you listen on ITunes or any other podcatcher, a review will help to grow the podcast!  Thank you in advance!

Re-Read Saturday News

The Re-Read Saturday focus on Eliyahu M. Goldratt and Jeff Cox’s The Goal: A Process of Ongoing Improvement began on February 21nd. The Goal has been hugely influential because it introduced the Theory of Constraints, which is central to lean thinking. The book is written as a business novel. Visit the Software Process and Measurement Blog and catch up on the re-read.

Note: If you don’t have a copy of the book, buy one.  If you use the link below it will support the Software Process and Measurement blog and podcast.

Dead Tree Version or Kindle Version 

Next . . . The Mythical Man-Month Get a copy now and start reading! We will start in two weeks!

Upcoming Events

Software Quality and Test Management 

September 13 ‚Äď 18, 2015

San Diego, California

http://qualitymanagementconference.com/

I will be speaking on the impact of cognitive biases on teams!  Let me know if you are attending!

More on other great conferences next week.

Next SPaMCast

The next Software Process and Measurement Cast will feature our essay on Managing Agile Projects and Teams (7/15 -7/20/13 and last week updates ‚Äď Meg). Teams make decisions daily that affect the direction of the sprint and project. The faster these decisions are made the higher the team‚Äôs velocity or productivity, and having a solid understanding of the real goals of the project helps the team make decisions more effectively.

We will also have columns from Kim Pries with his Software Sensei column and Gene Hughson with an entry from his Form Follows Function column.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques¬†co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: ‚ÄúThis book will prove that software projects should not be a tedious process, neither for you or your team.‚ÄĚ Support SPaMCAST by buying the book¬†here.

Available in English and Chinese.


Categories: Process Management

3 easy ways of improving your Kanban system

Xebia Blog - Sun, 06/14/2015 - 21:10

You are working in a Kanban team and after a few months it doesn’t seem to be as easy and effective as it used to be. Here are three tips on how to get that energy back up.

1. Recheck and refresh all policies

IMG_1884New team members haven‚Äôt gone through the same process as the other team members. This means that they probably won‚Äôt pick up on the subtleties of the board and policies that go with it. Over time team agreements that were very explicit (remember the Kanban Practice: ‚ÄúMake Policies Explicit‚ÄĚ?) aren‚Äôt anymore. For example: A blue stickie that used to have a class of service associated with it becomes a stickie for the person that works on the project. Unclear policies lead to loss of focus, confusion and lack of flow. Do a check if all policies are still valid, clear and understood by everybody. If not, get rid of them or improve them!

2. Check if WIP limits are understood and adhered to

In the first time after a team starts using Kanban, Work in Progress (WIP) limits are not completely optimized for flow. They usually are higher to minimize resistance from team and stakeholders as tighter WIP limits require a higher maturity of the organization. The team should start to adjust (lower) the limits to optimize flow. Lack team focus on improvement of flow can make that they don't adjust WIP limits.

Reevaluate current WIP limits, adjust them for bottlenecks and lower and discuss next steps. It will return the focus to help the team to ‚ÄúStart Finishing‚ÄĚ.

3. Do a complete overhaul of the board

IMG_0307It sounds counter intuitive, but sometimes it helps to take the board and do a complete runthrough of the steps to design a Kanban System. Analyze up- and downstream stakeholders and model the steps the work flows through. After that update the legend and policies for classes of service.

Even if the board looks almost the same, the discussion and analysis might lead to subtle changes that make a world of difference. And it’s good for team spirit!

 

I hope these tips can help you. Please share any that you have in the comments!

Re-Read Saturday: The Goal: A Process of Ongoing Improvement. Part 17

IMG_1249

 

Epiphanies, the profound and sudden flash of understanding, in any field are rare; however, when they do happen they generally transform the person that has the revelation but don‚Äôt always have an impact on those around them. Sitting on a plane from Los Angles this week as I prepared to write this entry of Re-read Saturday, the person next to me pointed at my copy of The Goal and stated ‚ÄúI read that book, it changed my life.‚ÄĚ As I have re-read The Goal I have been amazed at the number people that have stopped me to make sure I knew that they had read the book, also.¬† In today‚Äôs entry, the penultimate entry, Alex has an epiphany and discovers the answer to Johan‚Äôs final question to Alex.

Previous Installments:

Part 1       Part 2       Part 3      Part 4      Part 5 
Part 6       Part 7      Part 8     Part 9      Part 10
Part 11     Part 12      Part 13    Part 14    Part 15
Part 16

Chapter 39.  Bill Peach asks Alex to visit him and explain why his ideas are working.  Meanwhile, back at the plant all heck breaks loose.  The new orders have stressed the system.  Random problems in front of the bottleneck used up all of the buffer in front of the bottleneck steps.  The buffer had been used to keep he throughput through the bottleneck smooth and at maximum utilization. Once the buffer had been used up, the random problems caused work to get to the bottleneck in waves, causing the bottlenecks to be alternately starved and swamped.  This problem caused the plant to fall back to overtime and expediting work.

Alex leads a session with his team (soon to be Bob‚Äôs team) to find the root cause of the problem.¬† In the end the problem turned out to be that the new order had reduced the excess capacity in the plant.¬† Without changing the buffer in front of the bottleneck, the reduction in excess capacity meant that any shock to the system would quickly impact the bottlenecks which in turn would negatively impact the plants ability to deliver.¬† The team decides they need to rebuild the buffers to anticipate some level of shocks, using overtime and for the time being increase the delivery lead time.¬† Once a course was decided upon, Bob and ‚Äúhis‚ÄĚ team put it into action.¬† Chapter 39 marks the transition of Alex to being division manager and Bob to plant manager.

Chapter 40.  Alex, now division manager, and Lou, now Alex’s division comptroller, discuss the problems they have in front of them as they make the commute home from division headquarters. The problems at the division level are seemingly insurmountable. Examples they discuss include a delay introducing new product models (even though there is demand) so that organization does not have to mark down old product currently in inventory.  Lou and Alex decide to apply the five step process to the problem: however, both agree, that while the problem is important it not urgent enough to need to break family commitments that evening both men had.  This is unlike earlier in the book when Alex and his team would practically live in the plant.  In the vernacular of Agile, Alex has discovered a sustainable pace.

The five step process is:

  1. Find the constraint
  1. Exploit the constraint
  2. Subordinate every other step to the constraint
  3. Elevate the constraint, then
  4. Repeat if the constraint has been broken

When they reconvene in the morning they struggle with how to apply the five step process to the division.  They determine the constraints at the division level are the policies that drive the wrong behavior.  Both Alex and Lou struggle with how to apply the process when faced with concepts like policies rather than something tangible like a production process when the epiphany strikes.

Johan had left Alex with the task of answering, ‚ÄúWhat are the techniques needed for management?‚ÄĚ The struggle to apply the five step process lead Alex to the conclusion that, to manage, a leader must have the techniques to answer these questions:

  1. What to change?
  2. What to change to?
  3. How to cause the change?

Alex realizes he has learned to think for himself which was the outcome Johan had hoped for when he stopped providing advice.

Re-Read Saturday Notes:

  1. I anticipate that the re-read of The Goal will conclude next week with part 18. Our next book will be The Mythical Man-Month (I am buying a new copy today so if you do not have a copy . . . get a copy today and please use this version of Man-Month).
  2. Remember that the summary of previous entries in the re-read of The Goal have been shifted to a new page (here).
  3. Also, if you don’t have a copy of The Goal, buy one and read it!  If you use the link below it will support the Software Process and Measurement blog and podcast. Dead Tree Version or Kindle Version.

 


Categories: Process Management

The Willpower Instinct: Reducing time spent mindlessly scrolling for things to read

Mark Needham - Sat, 06/13/2015 - 00:12

I recently finished reading Kelly McGonigal’s excellent book ‘The Willpower Instinct‘ having previously watched her Google talk of the same title

My main takeaway from the book is that there are things that we want to do (or not do) but doing them (or not as the case may be) isn’t necessarily instinctive and so we need to develop some strategies to help ourselves out.

In one of the early chapters she suggests picking a habit that you want to do less off and write down on a piece of paper every time you want to do it and how you’re feeling at that point.

After writing it down you’re free to then follow through and do it but you don’t have to if you change your mind.

I was quite aware of the fact that I spend a lot of time idly scrolling from email to Twitter to Facebook to LinkedIn to news websites and back again so I thought it’d be interesting to track when/why I was doing this. The annoying thing about this habit is that it can easily eat up 20-30 minutes at a time without you even noticing.

I’ve been tracking myself for about three weeks and in the first few days I noticed that the first thing I did as soon as I woke up was grab my phone and get into the cycle.

It was quite frustrating to be lured in so early in the day but one of the suggestions in the book is that feeling guilty about something is actually detrimental to our progress. Instead we should note why it happened and then move on – the day isn’t a write off because of one event!

Kelly suggests that if we can work out the times when we’re most likely to fall into our habits then we can pre-plan a mitigation strategy.

From looking over my notes the following are the reasons why I want to start mindlessly scrolling:

  • I’m stuck on the problem I’m working on
  • I’m bored
  • I’m tired
  • I’m hungry
  • I’m getting distracted by notifications
  • I want to not think for a while

The notifications bullet is easy to address – I turn off notifications on my phone for 4 hours at a time so I don’t even know there’s anything to read.

I was intrigued to note that I got distracted when stuck on a problem – the main take away here is to check whether the urge to scroll mindlessly is being driven by having to think hard. If it is then I can choose to either get back to it or go for a short walk and then come back. But definitely don’t start scrolling!

I often find myself bored on my commute to work so I’ve addressed this by working out a book/paper I’m going to read the night before and then having that ready for the journey. Lunch time is prime time for mindless scrolling as well so I’ve filled that time with various computer science/data science videos.

Since I started tracking my scrolling I’ve found myself sleeping earlier so my assumption is that the extra hours awake were being spent mindlessly scrolling which led to being more tired so a win all around in that respect.

Something I’ve noticed is that I’m sometimes wasting time on other activities which I’m are not ‘forbidden’ but are equally unconstructive e.g. chat applications / watching music videos.

The former are obviously useful for communicating with people so I’ve been trying to use them only when I actually want to chat to someone rather than mindlessly looking for messages to read.

I also find myself not wanting to write down the times I’ve mindlessly scrolled when I’m doing it a lot on a given day. Being aware of this is helpful as I just write it down anyway and get on with the day.

The summary of my experience so far is it seems beneficial – I don’t think I’ve lost anything by not checking those mediums so often and I’ve definitely read a lot more than I usually do and been more focused as well.

Now I need to go and try out some of the other exercises from the book – if you’ve read it / tried out any of the tips I’d love to hear what’s worked well for you.

Categories: Programming

ATDD with Lego Mindstorms EV3

Xebia Blog - Fri, 06/12/2015 - 20:09

We have been building automated acceptance tests using web browsers, mobile devices and web services for several years now. Last week Erik Zeedijk and I came up with the (crazy) idea to implement features for a robot using ATDD. In this blog we will explain how we used ATDD while experimenting with Lego Mindstorms EV3.

What are we building?

When you practice ATDD it is really important to have a common understanding about what you need to build and test. So we had several conversations about requirements and specifications.

We wanted to build a robot that could help us clean up post-its from tables or floors after brainstorms and other sticky note related sessions. For our first iteration we decided to experiment with moving the robot and using the color scanner. The robot should search for yellow post-it notes all by itself. No manual interactions because we hate manual work. After the conversation we wrote down the following acceptance criteria to scope our development activities:

  • The robot needs to move his way on a white square board
  • The robot needs to turn away from the edges of the square board
  • The robot needs to stop a yellow post-it

We used Cucumber to write down the features and acceptance test scenarios like:


Feature: Move robot
Scenario: Robot turns when a purple line is detected
When the robot is moving and encounters a purple line
Then the robot turns left

Scenario: Robot stops moving when yellow post-it detected
When the robot is moving and encounters yellow post-it
Then the robot stops

Just enough information to start rocking with the robot! Ready? Yes! Tests? Yes! Lets go!

How did we build it?

First we played around with the Lego Mindstorms EV3 programming software (4GL tool). We could easily build the application by click our way through the conditions, loops and color detections.

But we couldn't find a way to hook up our Acceptance Tests. So we decided to look for programming languages. First hit was the leJOS. It comes with a custom firmware which you install on a SD card. This firmware allows you to run Java on the Lego EV3 brick.

Installing the firmware on the EV3 Brick

The EV3 main computer ‚ÄúBrick‚ÄĚ is actually pretty powerful.

It has a 300mhz ARM processor with 64MB RAM and takes micro SD cards for storage.
It also has Bluetooth and Wi-Fi making it possible to wirelessly connect to it and load programs on it.

There are several custom firmware’s you can run on the EV3 Brick. The LeJOS firmware allows us to do just that where we can program in Java with having access to a specialized API created for the EV3 Brick. Install it by following these steps:

  1. Download the latest firmware on sourceforge: http://sourceforge.net/projects/ev3.lejos.p/files/
  2. Get a blank SD card (2GB ‚Äď 32GB, SDXC not supported) and format this card to FAT32.
  3. Unzip the ‚Äėlejosimage.zip‚Äô file from the leJOS home directory to the root directory of the SD card.
  4. Download Java for Lego Mindstorms EV3 from the Oracle website: http://java.com/legomindstorms
  5. Copy the downloaded ‚ÄėOracle JRE.tar.gz‚Äô file to the root of the SD card.
  6. Safely remove the SD card from your computer, insert it into the EV3 Brick and boot the EV3. The first boot will take approximately 8 minutes while it created the needed partitions for the EV3 Brick to work.
ATDD with Cucumber

Cucumber helped us creating scenarios in an human readable language. We created the Step Definitions to glue it all the automation together and used the remote access to the EV3 via Java RMI.

We also wrote some support code to help us make the automation code readable and maintainable.

Below you can find an example of the StepDefinition of a Given statement of a scenario.

@Given("^the robot is driving$")
public void the_robot_is_driving() throws Throwable {
Ev3BrickIO.start();
assertThat(Ev3BrickIO.leftMotor.isMoving(), is(true));
assertThat(Ev3BrickIO.rightMotor.isMoving(), is(true));
}
Behavior Programming

A robot needs to perform a series of functions. In our case we want to let the robot drive around until he finds a yellow post-it, after which he stops. The obvious way of programming these functions is to make use of several if, then, else statements and loops. This works fine at first, however, as soon as the patterns become more complex the code becomes complex as well since adding new actions can have influence on other actions.

A solution to this problem is Behavior Programming. The concept of this is based around the fact that a robot can only do one action at a time depending on the state he is in. These behaviors are written as separate independent classes controlled by an overall structure so that behavior can be changed or added without touching other behaviors.

The following is important in this concept:

  1. One behavior is active at a time and then controls the robot.
  2. Each behavior is given a priority. For example; the turn behavior has priority over the drive behavior so that the robot will turn once the color sensor indicates he reaches the edge of the field.
  3. Each behavior is a self contained, independent object.
  4. The behaviors can decide if they should take control of the robot.
  5. Once a behavior becomes active it has a higher priority than any of the other behaviors
Behaviour API and Coding

The Behavior API is fairly easy to understand as it only exists out of one interface and one class. The individual behavior classes build-up is then defined by the overall Behavior interface and the individual behavior classes are controlled by an Arbitrator class that decides which behavior should be the active one. A separate class should be created for each behavior you want the robot to have.

leJOS Behaviour Pattern

 

When an Arbitrator object is instantiated, it is given an array of Behavior objects. Once it has these, the start() method is called and it begins arbitrating; deciding which behavior will become active.

In the code example below the Arbitrator is being created where the first two lines create instances of our behaviors. The third line places them into an array, with the lowest priority behavior taking the lowest array index. The fourth line creates the Arbitrator, and the fifth line starts the Arbitration process. One you create more behavior classes these can simply be added to the array in order to be executed by the Arbitrator.

DriveForward driveForward = new DriveForward();
Behavior stopOnYellow = new StopOnYellow();
behaviorList = new Behavior[]{driveForward, stopOnYellow};
arbitrator = new Arbitrator(behaviorList, true);
arbitrator.start();

We will now take a detailed look at one of the behavior classes; DriveForward(). Each of the standard methods is defined. First we create the action() method in which contains what we want the robot to perform when the behavior becomes active. In this case moving the left and the right motor forwards.

public void action() {
try {
suppressed = false;

Ev3BrickIO.leftMotor.forward();
Ev3BrickIO.rightMotor.forward();
while( !suppressed ) {
Thread.yield();
}

Ev3BrickIO.leftMotor.stop(true);
Ev3BrickIO.rightMotor.stop(true);
} catch (RemoteException e) {
e.printStackTrace();
}
}

The suppress() method will stop the action once this is called.

public void suppress() {
suppressed = true;
}

The takeControl() method tells the Arbitrator which behavior should become active. In order to keep the robot moving after having performed more actions we decided to create a global variable to keep track of the fact if the robot is running. While this is the case we simply return 'true' from the takeControl() method.

public boolean takeControl() {
return Ev3BrickIO.running;
}

You can find our leJOS EV3 Automation code here.

Test Automation Day

Our robot has the potential to do much more and we have not yet implemented the feature of picking up our post-its! Next week we'll continue working with the Robot during the Test Automation Day.
Come and visit our booth where you can help us to enable the Lego Mindstorms EV3 Robot to clean-up our sticky mess at the office!

Feel free to use our Test Automation Day ‚ā¨50 discount on the ticket price by using the following code: TAD15-XB50

We hope to see you there!

Updates to Unity, C++, and iOS tools for Play game services

Android Developers Blog - Fri, 06/12/2015 - 17:45

Posted by Benjamin Frenkel, Product Manager

To further support all you game developers, we've updated our popular developer tools to give you a consistent set of game services across platforms for a better, more stable experience, with a particular focus on improvements to the Play game services Unity plugin. In addition, we added support for the Nearby Connections API, launched earlier this year at GDC, to our C++ SDK and Unity plugin.

Let’s take a look a closer look!

Unity plugin feature parity and stability improvements

We’ve added full support for Events and Quests in the Unity plugin. If you’re a Unity developer, you can now incorporate Quests into your games and take full advantage of Player Analytics natively within the Unity IDE.

We’ve also listened to feedback from our community of Unity plugin users and made stability improvements to Play game services Multiplayer, Saved Games, and to sign-in. You’ll now have a much better experience integrating with these Play game services, with fewer crashes and glitches.

C++ SDK and Unity support for the Nearby Connections API

We have added support for the Nearby Connections API to our C++ SDK and Unity plugin. You can now easily build awesome second screen and local multiplayer experiences, like this Beach Bugging Racing example, with the development tools you are most comfortable with.

Easier and more stable iOS builds with CocoaPods

We’ve also made major improvements to our Play game services CocoaPods, which simplify dependency management and building App Store packages from Xcode. The CocoaPods will improve building for iOS with the Play game services iOS and C++ SDKs, and the Unity plugin. We also improved the stability of multiplayer on iOS, eliminating many of the issues around accepting match invitations.

Finally, we improved our support for iOS 8, making it easier to set up multiplayer push notifications, and fixing UI compatibility issues.

Quick links to get you started

Play game services developer page: https://developers.google.com/games/services/
Case studies: http://developer.android.com/distribute/stories/games.html

Downloads

Join the discussion on

+Android Developers
Categories: Programming

Unit and integration are ambiguous names for tests

Coding the Architecture - Simon Brown - Fri, 06/12/2015 - 17:10

I've blogged about architecturally-aligned testing before, which essentially says that our automated tests should be aligned with the constructs that we use to describe a software system. I want to expand upon this topic and clarify a few things, but first...

Why?

In a nutshell, I think the terms "unit test" and "integration test" are ambiguous and open to interpretation. We tend to all have our own definition, but I've seen those definitions vary wildly, which again highlights that our industry often lacks a shared vocabulary. What is a "unit"? How big is a "unit"? And what does an "integration test" test the integration of? Are we integrating components? Or are we integrating our system with the database? The Wikipedia page for unit testing talks about individual classes or methods ... and that's good, so why don't we use a much more precise terminology instead that explicitly specifies the scope of the test?

Align the names of the tests with the things they are testing

I won't pretend to have all of the answers here, but aligning the names of the tests with the things they are testing seems to be a good starting point. And if you have a nicely defined way of describing your software system, the names for those tests fall into place really easily. Again, this comes back to creating that shared vocabulary - a ubiquitous language for describing the structures at different levels of abstraction within a software system.

Here are some example test definitions for a software system written in Java or C#, based upon my C4 model (a software system is made up of containers, containers contain components, and components are made up of classes).

Name What? Test scope? Mocking, stubbing, etc? Lines of production code tested per test case? Speed Class tests Tests focused on individual classes, often by mocking out dependencies. These are typically referred to as ‚Äúunit‚ÄĚ tests. A single class - one method at a time, with multiple test cases to test each path through the method if needed. Yes; classes that represent service providers (time, random numbers, key generators, non-deterministic behaviour, etc), interactions with ‚Äúexternal‚ÄĚ services via inter-process communication (e.g. databases, messaging systems, file systems, other infrastructure services, etc ... the adapters in a ‚Äúports and adapters‚ÄĚ architecture). Tens Very fast running; test cases typically execute against resources in memory. Component tests Tests focused on components/services through their public interface. These are typically referred to as ‚Äúintegration‚ÄĚ tests and include interactions with ‚Äúexternal‚ÄĚ services (e.g. databases, file systems, etc). See also ComponentTest on Martin Fowler's bliki. A component or service - one operation at a time, with multiple test cases to test each path through the operation if needed. Yes; other components. Hundreds Slightly slower; component tests may incur network hops to span containers (e.g. JVM to database). System tests UI, API, functional and acceptance tests (‚Äúend-to-end‚ÄĚ tests; from the outside-in). See also BroadStackTest on Martin Fowler's bliki. A single scenario, use case, user story, feature, etc across a whole software system. Not usually, although perhaps other systems if necessary. Thousands Slower still; a single test case may require an execution path through multiple containers (e.g. web browser to web server to database).

Thoughts?

This definition of the tests doesn't say anything about TDD, when you write your tests and how many tests you should write. That's all interesting, but it's independent to the topic we're discussing here. So then, do *you* have a good clear definition of what "unit testing" means? Do your colleagues at work agree? Can you say the same for "integration testing"? What do you think about aligning the names of the tests with the things they are testing? Thoughts?

Categories: Architecture

Time for a Speaking Break, and a Writing Streak

NOOP.NL - Jurgen Appelo - Fri, 06/12/2015 - 10:33
speaking break

When I travel a lot, I dream of being back home for a while so that I can “get things done”. My God, the traveling is a giant productivity killer!

However, when I am home for a couple of months, I yearn for new events, conferences, and gatherings. Because, even for an introvert like me, working alone can feel so… alone.

The post Time for a Speaking Break, and a Writing Streak appeared first on NOOP.NL.

Categories: Project Management

Software Engineering Radio - Episode 228

Coding the Architecture - Simon Brown - Fri, 06/12/2015 - 10:23

A quick note to say that my recent interview/chat with Sven Johann for Software Engineering Radio has now been published. As you might expect, it covers software architecture, diagramming, the use of UML, my C4 model, doing "just enough" and why the code should (but doesn't often) reflect the software architecture. You can listen to the podcast here. Enjoy!

Categories: Architecture

Agile Project Management: The Tipping Point (Refined)

One Side Or The Other!

One Side Or The Other!

As organizations transition to Agile, teams often ask about how to change from classic command and control project management methods into self-organizing teams. Self-organizing and self-managing teams do not just appear magically. Both the project teams and the organization have baggage that needs to be rationalized and unlearned to progress from needing to be lead to leading themselves. The concepts which have to be learned include the following: measuring business value, team decision-making and sharing the real goals of projects.

Classic command and control project management (which I would classify as minimally collaborative) is based on the premise that the project manager has both unique knowledge and a unique set of skills not held by members of the team.¬† These skills are thought to revolve around project planning, controlling the people, office politics and status reporting. Certifications, like the PMP¬ģ, have been used to encourage these belief.¬† In this environment the measures of success for projects are often delivering on-time, on-budget and on-scope.

In Agile projects, as discussed in Project Management is Dead, the roles of the classic project manager in Scrum are spread across the three basic roles (Product Owner, Scrum Master and Development Team). A fourth role, the Agile Program Manager (known as a Release Train Engineer in SAFe), is needed when multiple projects are joined together to become a coordinated program.  The primary measures of success in Agile projects are delivered business value and customer satisfaction.  These attributes subsume the classic topics of on-time, on-budget and on-scope. (Note: Delivered value and customer satisfaction should be the primary measure of success in ALL types of projects, however these are not generally how project teams are held accountable.)

As teams learn to embrace and use Agile principles, they need to learn how to make decisions as a team. The decisions that teams need to learn how to make for themselves always have consequences, and sometimes those consequences will be negative. To accomplish this learning process in the least risky manner, the team should use techniques like delaying decisions as late as is practical and delivering completed work within short time boxes. These techniques reduce risk by increasing the time the team has to gather knowledge and by getting the team feedback quickly. The organization also must learn how to encourage the team to make good decisions while giving them the latitude to mess up. This requires the organization to accept some level of explicit initial ambiguity that is caused by delaying decisions, rather than implicit ambiguity of making decisions early that later turn out to be wrong. The organization must also learn to evaluate teams and individuals less on the outcome of a single decision and more on the outcome of the value the team delivered.

Teams, on the other hand, have to unlearn bad habits. For example, teams need to unlearn the habit of relying on others to plan for them. In order to do that, all leaders and teams must have an understanding of the true goals of the project (listen to my interview with David Marquet) and how the project fits into the strategic goals of the organization.

Teams make decisions daily that affect the direction of the sprint and project. The faster these decisions are made the higher the team’s velocity or productivity, and having a solid understanding of the real goals of the project helps the team make decisions more effectively. Organizations need to learn how share knowledge that today is generally compartmentalized between developers, testers or analysts.

The process of learning and unlearning occurs on a continuum as teams push toward a type of collective self-actualization. As any team moves toward its full potential, the organization’s need to control planning and decisions falls away. If the organization doesn’t back away from the tenants of command and control and move towards the Agile principles, the ability of any team to continue to grow will be constrained.  The tipping point generally occurs when an organization realizes that self-managing and self-organizing teams deliver superior value and higher customer satisfaction and that in the long run is what keeps CIOs employed.


Categories: Process Management

Neo4j: Using LOAD CSV to help explore CSV files

Mark Needham - Fri, 06/12/2015 - 00:15

During the Neo4j How I met your mother hackathon that we ran last week one of the attendees noticed that one of the CSV files we were importing wasn’t creating as many records as they expected it to.

This is typically the case when there’s some odd quoting in the CSV file but we decided to look into it.

The file in question was one containing references made in HIMYM. The first 5 lines look like this:

$ head -n 5 data/import/references.csv
ReferencedEpisodeId,ReferencingEpisodeId,ReferenceText
168,184,"Marshall will eventually hear back from the New York State Judicatory Committee in Something New, which will become a main plot point of Season 9."
168,169,Barney proclaiming to be done with Robin will be the focal point of Lobster Crawl.
58,57,"Barney finally confronts his saboteur (Abby, whom he slept with in Ten Sessions) in Everything Must Go."
58,63,"Barney finally confronts his saboteur (Abby, whom he slept with in Ten Sessions) in Everything Must Go."

And this is how many lines the Unix ‘wc’ command sees:

$ wc -l data/import/references.csv
     782 data/import/references.csv

So we might expect that there are going to be 782 records created if we import that file into Neo4j. Let’s run a quick query in Neo4j to see what it thinks:

LOAD CSV WITH HEADERS 
FROM "file:///Users/markneedham/projects/neo4j-himym/data/import/references.csv" 
AS row
return count(*)
 
==> +----------+
==> | count(*) |
==> +----------+
==> | 636      |
==> +----------+
==> 1 row

So we have 146 less records than we expected which means Neo4j is treating multiple lines as one CSV line in some cases.

Let’s go back to the Unix command line to try and work out which lines those are. There must be some lines which start with part of the ‘ReferenceText’ rather than a ‘ReferenceEpisodeId’ so let’s extract the first column and see what’s going on there:

$ cat data/import/references.csv | cut -d"," -f1 | grep -v  '[0-9]\+$'| head -n 10
ReferencedEpisodeId
This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny
This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny
This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny
This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny
This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny
This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny
This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny
This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny
Also

We’ve extracted the first column and then filter the output to only keep rows which don’t contain all numbers which will be our rogue rows.

Let’s switch back to Neo4j land to see which rows it thinks contains these fragments of text:

LOAD CSV WITH HEADERS FROM "file:///Users/markneedham/projects/neo4j-himym/data/import/references.csv" AS row
WITH row WHERE row.ReferenceText =~ ".*This is the Mother's first.*"
RETURN row.ReferencedEpisodeId, row.ReferencingEpisodeId, row.ReferenceText
 
==> +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> | row.ReferencedEpisodeId | row.ReferencingEpisodeId | row.ReferenceText                                                                                                                                                                                                                                                                                                                                                                     |
==> +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> | "45"                    | "37"                     | "This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9." |
==> | "45"                    | "184"                    | "This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9." |
==> +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Interestingly it only returns two rows containing that phrase whereas we see it at least 8 times. Initially I thought this was an issue with the LOAD CSV command but if we filter the rows to only return ones that have a ‘ReferencedEpisodeId’ of ’45’ then we do see them returned:

==> +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> | row                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
==> +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "53", ReferenceText -> "The website counting down to the next slap (slapcountdown.com) that Marshall sends Barney reaches zero in Slapsgiving, when the third slap is delivered."}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "55", ReferenceText -> "Ted gets rid of his butterfly tramp stamp through ten weekly sessions of laser tattoo removal between The Platinum Rule and Ten Sessions, over the course of which he meets, asks out, and eventually starts dating his dermatologist, Stella Zinman."}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "57", ReferenceText -> "Ted gets rid of his butterfly tramp stamp through ten weekly sessions of laser tattoo removal between The Platinum Rule and Ten Sessions, over the course of which he meets, asks out, and eventually starts dating his dermatologist, Stella Zinman."}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "56", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."}  |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "200", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."} |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "100", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."} |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "86", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."}  |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "113", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."} |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "161", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."} |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "37", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."}  |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "184", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."} |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "37", ReferenceText -> "This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."}                                                                                                                                                                                                                                                                                                                                                                         |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "184", ReferenceText -> "This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."}                                                                                                                                                                                                                                                                                                                                                                        |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "63", ReferenceText -> "Marshall makes other home-made websites in Everything Must Go (lilyandmarshallselltheirstuff.com) and The Sexless Innkeeper (itwasthebestnightever.com), where Lily and Future Ted mention it being a problem."}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "92", ReferenceText -> "Marshall makes other home-made websites in Everything Must Go (lilyandmarshallselltheirstuff.com) and The Sexless Innkeeper (itwasthebestnightever.com), where Lily and Future Ted mention it being a problem."}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
==> +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

So the actual problem is that the regex matcher doesn’t deal with the new line in the string.

Our next step is therefore to get rid of new lines within strings. I spent ages trying to find the appropriate command before coming across the following use of awk which does the job:

$ cat data/import/references.csv | awk '(NR-1)%2{$1=$1} {print $0}' RS=\" ORS=\" | wc -l
637
 
$ cat data/import/references.csv | awk '(NR-1)%2{$1=$1} {print $0}' RS=\" ORS=\" > data/import/refs.csv

Let’s try the LOAD CSV command again:

LOAD CSV WITH HEADERS FROM "file:///Users/markneedham/projects/neo4j-himym/data/import/refs.csv" AS row
WITH row WHERE row.ReferenceText =~ ".*This is the Mother's first.*"
RETURN row.ReferencedEpisodeId, row.ReferencingEpisodeId, row.ReferenceText
 
==> +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> | row.ReferencedEpisodeId | row.ReferencingEpisodeId | row.ReferenceText                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
==> +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> | "45"                    | "56"                     | "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton. This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9." |
==> | "45"                    | "200"                    | "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton. This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9." |
==> | "45"                    | "100"                    | "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton. This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9." |
==> | "45"                    | "86"                     | "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton. This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9." |
==> | "45"                    | "113"                    | "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton. This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9." |
==> | "45"                    | "161"                    | "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton. This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9." |
==> | "45"                    | "37"                     | "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton. This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9." |
==> | "45"                    | "184"                    | "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton. This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9." |
==> | "45"                    | "37"                     | "This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."                                                                                                                                                                                                                                                                                                                                                                        |
==> | "45"                    | "184"                    | "This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."                                                                                                                                                                                                                                                                                                                                                                        |
==> +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

And there we go!

Update

Michael pointed out that I could have used the dotall regex flag at the beginning of the regular expression in order to search across new lines without having to remove them! In that case the query would read like this:

LOAD CSV WITH HEADERS FROM "file:///Users/markneedham/projects/neo4j-himym/data/import/references.csv" AS row
WITH row WHERE row.ReferenceText =~ "(?s).*This is the Mother.*"
RETURN row
 
==> +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> | row                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
==> +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "56", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."}  |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "200", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."} |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "100", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."} |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "86", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."}  |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "113", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."} |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "161", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."} |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "37", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."}  |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "184", ReferenceText -> "Ted eventually acquires the yellow umbrella in No Tomorrow (after the Mother leaves it behind at the St. Patrick's Day party, as seen in How Your Mother Met Me), and leaves it in Cindy's and the Mother's apartment in Girls Versus Suits. The umbrella is also seen/referenced in many other episodes, including Right Place, Right Time, Big Days, and Farhampton.
==> This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."} |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "37", ReferenceText -> "This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."}                                                                                                                                                                                                                                                                                                                                                                         |
==> | {ReferencedEpisodeId -> "45", ReferencingEpisodeId -> "184", ReferenceText -> "This is the Mother's first on-screen appearance with the yellow umbrella. Previously she appeared in Lucky Penny, with her head obscured by a bridal veil. She is seen again in No Tomorrow, again hidden by the umbrella, her ankle is seen briefly in Girls Versus Suits, and she gets her first proper appearance in Something New, after which she appears throughout Season 9."}                                                                                                                                                                                                                                                                                                                                                                        |
==> +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Categories: Programming

Mac OS X: GNU sed ‚Äď Hex string replacement / replacing new line characters

Mark Needham - Thu, 06/11/2015 - 22:38

Recently I was working with a CSV file which contained both Windows and Unix line endings which was making it difficult to work with.

The actual line endings were HEX ‘0A0D’ i.e. Windows line breaks but there were also HEX ‘OA’ i.e. Unix line breaks within one of the columns.

I wanted to get rid of the Unix line breaks and discovered that you can do HEX sequence replacement using the GNU version of sed – unfortunately the Mac ships with the BSD version which doesn’t have this functionaltiy.

The first step was therefore to install the GNU version of sed.

brew install coreutils
brew install gnu-sed --with-default-names

I wanted to replace my system sed so that’s why I went with the ‘–with-default-names’ flag – without that flag I believe the sed installation would be accessible as ‘gs-sed’.

The following is an example of what the lines in the file look like:

$ echo -e "Hello\x0AMark\x0A\x0D"
Hello
Mark

We want to get rid of the new line in between ‘Hello’ and ‘Mark’ but leave the other one be. I adapted one of the commands from this tutorial to look for lines which end in ‘0A’ where that isn’t followed by a ‘0D':

$ echo -e "Hello\x0AMark\x0A\x0D" | \
  sed 'N;/\x0A[^\x0D]/s/\n/ /'
Hello Mark

Let’s go through the parts of the sed command:

  • N – this creates a multiline pattern space by reading a new line of input and appending it to the contents of the pattern space. The two lines are separated by a new line.
  • /\x0A[^\x0D]/ – this matches any lines which contain ‘OA’ not followed by ‘OD’
  • /s/\n/ / – this substitutes the new line character with a space for those matching lines from the previous command.

Now let’s check it works if we have multiple lines that we want to squash:

$ echo -e "Hello\x0AMark\x0A\x0DHello\x0AMichael\x0A\x0D"
Hello
Mark
Hello
Michael
 
$ echo -e "Hello\x0AMark\x0A\x0DHello\x0AMichael\x0A\x0D" | \
  sed 'N;/\x0A[^\x0D]/s/\n/ /'
Hello Mark
Hello Michael

Looks good! The actual file is a bit more nuanced so I’ve still got a bit more work to do but this is a good start.

Categories: Programming

Android Developer Story: Trello Increases engagement with material design

Android Developers Blog - Thu, 06/11/2015 - 21:29

Posted by Laura Della Torre, Google Play team

Trello is a visual collaboration tool that gives teams a shared perspective on projects. It’s built around the concept of a traditional office whiteboard. Simplicity and flexibility are core to the product, so the Trello team recently redesigned their Android app using the material design guidelines to double down on that effort.

According to Fyza Hashim, Designer at Trello, material design had an immediate impact on streamlining app-design and -development at the company. She added that, ‚ÄúBecause the guidelines are so thorough and well thought out, you don‚Äôt have to go back and forth with developers.‚ÄĚ

Sharing is a key component of Trello, so material design helped continue the same cohesive design and intuitive experience on both web and mobile. This makes sharing even easier. As a result, Trello has also seen double digit growth in user engagement with more and more sessions added per week.

Watch the video where we caught up with Michael Pryor, CEO; Hamid Palo, Mobile Lead; and Fyza at the Trello offices in New York to learn more.

Material design ‚ÄĒ learn more about material design and how it helps you create beautiful, engaging apps.

Join the discussion on

+Android Developers
Categories: Programming

Is Paying for Developer Bootcamp Worth It?

Making the Complex Simple - John Sonmez - Thu, 06/11/2015 - 16:00

In this episode, I share the pros and cons about paying for a developer bootcamp. Full transcript: John:¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬† Hey, this is John Sonmez from simpleprogrammer.com. I‚Äôve got another question here about mobile development, actually about training which‚ÄĒI think this is kind of interesting. This question comes from Jason. Jason says, ‚ÄúHi John. I have been […]

The post Is Paying for Developer Bootcamp Worth It? appeared first on Simple Programmer.

Categories: Programming

Diagrams for System Evolution

Coding the Architecture - Simon Brown - Thu, 06/11/2015 - 08:17

Every system has an architecture (whether deliberately designed or not) and most systems change and evolve during their existence - which includes their architecture. Sometimes this happens gradually and sometimes in large jumps. Let's have a look at a simple example.

Basic context diagram for a furniture company


Before


The main elements are:

  • Sales Orderbook where sales staff book orders and designers occasionally view sales.
  • Product Catalogue where designers record the details of new products.
  • Warehouse Management application that takes orders and either delivers to customers from stock or forward orders to manufacturing.


The data flow is generally one way, with each system pushing out data with little feedback, due to an old batch-style architecture. For example, the designer updates product information but all the updates for the day are pushed downstream at end-of-day. The Warehouse application only accepts a feed from the Sales Orderbook, which includes both orders and product details.


There are multiple issues with a system like this:

  1. Modification to the products (new colours, sizes, materials etc) are not delivered to the manufacturer until after an order is placed. This means a long lead time for manufacturing and an unhappy customer.
  2. The sales orderbook receives a subset of the product information when updated. This means the information could become out-of-sync and not all of it is available.
  3. The warehouse system also relies on the sales orderbook for product information. This may be out of date or inaccurate.
  4. The designer does not have easy-to-access information about how product changes affect sales.


The users are unhappy with the system and want the processes to change as follows:

  1. The product designer wishes to see historic order information alongside the products in the catalog. Therefore the catalog should be able to retrieve data from the Sales Orderbook when required.
  2. Don't batch push products to the Orderbook and warehouse management. They should access the product catalog as a service when required.


System redesign

Therefore the system is re-architected as follows:



This is a significant modification to how the systems works:

  • The Sales Orderbook and Product Catalogue have a bi-directional relationship and request information from each other.
  • Only orders are sent from the Sales Orderbook to the Product Catalogue.
  • There is now a direct connection between the Warehouse Manager and the Product Catalogue.
  • The product catalogue now provides all features the Designer requires. They do not need to access any other interfaces into the system.

However, if you place the two diagrams next to each other they look similar and the scale of the change is not obvious.

Before

When we write code, we would use a ‚Äėdiff‚Äô tool to show us the modifications.


Before


I often do something similar on my architecture diagrams. For example:


System before modification:

Before


System after modification:


When placed side-by-side the differences are much more obvious:

Before


I’ve used similar colours to those used by popular diff tools; with red for a removed relationship and green for a new relationship (I would use the same colours if components were added or removed). I’ve used blue to indicate where a relationship has changed significantly. Note that I’ve included a Key on these diagrams.

You may be thinking; why have I coloured before and after diagrams rather than having a single, coloured diagram? A single diagram can work if you are only adding and removing elements but if they are being modified it becomes very cluttered. As an example, consider the relationship between the Sales Orderbook and the Product Catalogue. Would the description on a combined diagram be "product updates", "exchange sales and product information", or both? How would I show which was before and after? Lastly, what would I do with the arrow on the connection? Colourise it differently from the line?

Conclusion

I find that having separate, coloured diagrams for the current and desired system architectures allows the changes to be seen much more clearly. Having spoken to several software architects, it appears that many use similar techniques, however there appears to be no standard way to do this. Therefore we should make sure that we follow standard advice such as using keys and labelling all elements and connections.

Categories: Architecture

Episode 228: Software Architecture Sketches with Simon Brown

Simon Brown, author of Software Architecture for Developers, talks with Sven Johann¬†about using simple sketches to visualize, communicate and document software architecture. The show starts with a consideration of why sketches are more useful than UML and then continues with the explanation of the C4 model (context, containers, components and classes) and how it fits […]
Categories: Programming

Interview with Dave Gray about The Connected Company

NOOP.NL - Jurgen Appelo - Wed, 06/10/2015 - 13:55
the-connected-company

Last year, I read the book The Connected Company by Dave Gray, and I gave it five stars. As an experiment, I decided to reach out to Dave with a couple of questions, and he graciously took time to reply.

The post Interview with Dave Gray about The Connected Company appeared first on NOOP.NL.

Categories: Project Management

Project Management Is Dead (Refined)

An Eternal Flame.

An Eternal Flame.

In Mastering Software Project Management (J Ross Publishing 2010), Murali Chemuturi and I define software project management as the activities required to plan and lead software projects.  Historically, IT projects and most non-Agile frameworks have identified a single person to play this role.  Programs, which are made up multiple projects, include multiple project managers that report to program manager. However, many forms of Agile now have eschewed the role of the project manager and instead distributed the activities associated with project management across the core team, including the product owner, the development team and the Scrum Master. Project management as a role is dead, long live project management the concept.

The product owner is responsible for managing a number of the activities that the project manager or administrator would have been tasked with in the past.  The primary role of the product owner is to own and manage the product backlog. Managing the backlog includes the activities of prioritizing backlog items and determining the release plan (including scope and date).  Managing the backlog in many organizations also means the product owner manages the budget, communicating progress, and interacts with external the stakeholders.  The product owner acts as a leader, providing the team with a direction using the backlog. He or she manages the backlog as a tool to exhibit that leadership.

The development team members also pick up some of the project management tasks.  The development team is responsible for identifying and managing the tasks needed to deliver the work they have committed to complete. The development team roles mix creation and innovation with control and management, using techniques such as peer pressure, stand-up meetings and pair working.

The Scrum Master is responsible for facilitating, coaching and motivating the team.  Scrum Masters guide teams so that they learn and use Agile techniques, confront delivery problems as they occur and work together as a well-oiled unit. The Scrum Master also serves as a shepherd to stave off interference from outside the team’s boundaries. The Scrum Master interacts with a team or teams, and then let the team members synthesize and internalize the advice. The Scrum Master is the team’s tactical coach.

In Agile, project management is dead, at least as a single role that leads, directs, controls and administers a project team because those roles are distributed to the team. I was once asked, “In an Agile project, who is the single person I can put my foot on their throat to motivate?” ¬†Without dignifying the question, in an Agile environment the answer is far less obvious than pointing to a project manager. ¬†The role simply isn‚Äôt filled by a single project manager. Now these responsibilities and tasks are distributed to those who that are actually have both the authority and responsibility for project execution.


Categories: Process Management

Sponsored Post: Datadog, Tumblr, Power Admin, Learninghouse, MongoDB, Internap, Aerospike, SignalFx, InMemory.Net, Couchbase, VividCortex, MemSQL, Scalyr, AiScaler, AppDynamics, ManageEngine, Site24x7

Who's Hiring?
  • Make Tumblr fast, reliable and available for hundreds of millions of visitors and tens of millions of users.  As a Site Reliability Engineer you are a software developer with a love of highly performant, fault-tolerant, massively distributed systems. Apply here now! 

  • At Scalyr, we're analyzing multi-gigabyte server logs in a fraction of a second. That requires serious innovation in every part of the technology stack, from frontend to backend. Help us push the envelope on low-latency browser applications, high-speed data processing, and reliable distributed systems. Help extract meaningful data from live servers and present it to users in meaningful ways. At Scalyr, you’ll learn new things, and invent a few of your own. Learn more and apply.

  • UI EngineerAppDynamics, founded in 2008 and lead by proven innovators, is looking for a passionate UI Engineer to design, architect, and develop our their user interface using the latest web and mobile technologies. Make the impossible possible and the hard easy. Apply here.

  • Software Engineer - Infrastructure & Big DataAppDynamics, leader in next generation solutions for managing modern, distributed, and extremely complex applications residing in both the cloud and the data center, is looking for a Software Engineers (All-Levels) to design and develop scalable software written in Java and MySQL for backend component of software that manages application architectures. Apply here.
Fun and Informative Events
  • 90 Days. 1 Bootcamp. A whole new life. Interested in learning how to code? Concordia St. Paul's Coding Bootcamp is an intensive, fast-paced program where you learn to be a software developer. In this full-time, 12-week on-campus course, you will learn either .NET or Java and acquire the skills needed for entry-level developer positions. For more information, read the Guide to Coding Bootcamp or visit bootcamp.csp.edu.

  • June 2nd – 4th, Santa Clara: Register for the largest NoSQL event of the year, Couchbase Connect 2015, and hear how innovative companies like Cisco, TurboTax, Joyent, PayPal, Nielsen and Ryanair are using our NoSQL technology to solve today’s toughest big data challenges. Register Today.

  • The Art of Cyberwar: Security in the Age of Information. Cybercrime is an increasingly serious issue both in the United States and around the world; the estimated annual cost of global cybercrime has reached $100 billion with over 1.5 million victims per day affected by data breaches, DDOS attacks, and more. Learn about the current state of cybercrime and the cybersecurity professionals in charge with combatting it in The Art of Cyberwar: Security in the Age of Information, provided by Russell Sage Online, a division of The Sage Colleges.

  • MongoDB World brings together over 2,000 developers, sysadmins, and DBAs in New York City on June 1-2 to get inspired, share ideas and get the latest insights on using MongoDB. Organizations like Salesforce, Bosch, the Knot, Chico’s, and more are taking advantage of MongoDB for a variety of ground-breaking use cases. Find out more at http://mongodbworld.com/ but hurry! Super Early Bird pricing ends on April 3.
Cool Products and Services
  • Datadog is a monitoring service for scaling cloud infrastructures that bridges together data from servers, databases, apps and other tools. Datadog provides Dev and Ops teams with insights from their cloud environments that keep applications running smoothly. Datadog is available for a 14 day free trial at datadoghq.com.

  • Here's a little quiz for you: What do these companies all have in common? Symantec, RiteAid, CarMax, NASA, Comcast, Chevron, HSBC, Sauder Woodworking, Syracuse University, USDA, and many, many more? Maybe you guessed it? Yep! They are all customers who use and trust our software, PA Server Monitor, as their monitoring solution. Try it out for yourself and see why we’re trusted by so many. Click here for your free, 30-Day instant trial download!

  • Turn chaotic logs and metrics into actionable data. Scalyr replaces all your tools for monitoring and analyzing logs and system metrics. Imagine being able to pinpoint and resolve operations issues without juggling multiple tools and tabs. Get visibility into your production systems: log aggregation, server metrics, monitoring, intelligent alerting, dashboards, and more. Trusted by companies like Codecademy and InsideSales. Learn more and get started with an easy 2-minute setup. Or see how Scalyr is different if you're looking for a Splunk alternative or Loggly alternative.

  • Instructions for implementing Redis functionality in Aerospike. Aerospike Director of Applications Engineering, Peter Milne, discusses how to obtain the semantic equivalent of Redis operations, on simple types, using Aerospike to improve scalability, reliability, and ease of use. Read more.

  • SQL for Big Data: Price-performance Advantages of Bare Metal. When building your big data infrastructure, price-performance is a critical factor to evaluate. Data-intensive workloads with the capacity to rapidly scale to hundreds of servers can escalate costs beyond your expectations. The inevitable growth of the Internet of Things (IoT) and fast big data will only lead to larger datasets, and a high-performance infrastructure and database platform will be essential to extracting business value while keeping costs under control. Read more.

  • SignalFx: just launched an advanced monitoring platform for modern applications that's already processing 10s of billions of data points per day. SignalFx lets you create custom analytics pipelines on metrics data collected from thousands or more sources to create meaningful aggregations--such as percentiles, moving averages and growth rates--within seconds of receiving data. Start a free 30-day trial!

  • InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net

  • VividCortex goes beyond monitoring and measures the system's work on your MySQL and PostgreSQL servers, providing unparalleled insight and query-level analysis. This unique approach ultimately enables your team to work more effectively, ship more often, and delight more customers.

  • MemSQL provides a distributed in-memory database for high value data. It's designed to handle extreme data ingest and store the data for real-time, streaming and historical analysis using SQL. MemSQL also cost effectively supports both application and ad-hoc queries concurrently across all data. Start a free 30 day trial here: http://www.memsql.com/

  • aiScaler, aiProtect, aiMobile Application Delivery Controller with integrated Dynamic Site Acceleration, Denial of Service Protection and Mobile Content Management. Also available on Amazon Web Services. Free instant trial, 2 hours of FREE deployment support, no sign-up required. http://aiscaler.com

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below. Please click to read more...

Categories: Architecture