Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Architecture

Are you an IT project manager?

Coding the Architecture - Simon Brown - Tue, 01/28/2014 - 21:59

Although you won't generally find me talking about IT project management, it is a topic that overlaps with software architecture in a number of areas. For example, it's fairly typical to see software teams where a software architect and project manager are jointly responsible for the successful delivery of a software system. Here, the software architect is usually responsible for the technical risks and the project manager picks up all of the "other stuff" (e.g. risks associated with timescales, budget, scope, etc). Project management is an interesting area, particularly when you start throwing agile approaches into the mix. In the same way that I've discovered a lot of uncertainty about the software architecture role, my wife has discovered the same of project management during her career. And this has inspired her to write a book. To quote from start of it:

My background is in software development and, having made the decision to become more involved with managing projects, I noticed that most websites, articles and books discuss project management from a very generic perspective. There seem to be very few books that take a more holistic view of IT project management including what skills you need, problems you may come up against due to technology and general tips on project management principles. After all, technology projects have their own unique activities and challenges.

Now working within a project management office, I provide guidance and support services for project managers who work within a complex and changing IT environment. In undertaking this role, I found that the advice, support and document reviews I undertook started to show some common trends. These include a lack of understanding of how to define and present IT projects to the business and common mistakes made in day-to-day project management.

This trend led to me sounding like a “broken record”, repeating the same advice over and over again to different project managers. That started me thinking about writing a short book that is clear, concise and quick to read, covering the skills, principles, tools and activities that will help you to successfully manage an IT project in the real world.

Are you an IT project manager?

The book is currently titled "Are you an IT project manager?" and is a work in progress, being published incrementally on Leanpub. I may be biased, but I'm looking forward to it evolving over the coming weeks, particularly as I've heard some of the stories that are yet to be written about.

Categories: Architecture

How Next Big Sound Tracks Over a Trillion Song Plays, Likes, and More Using a Version Control System for Hadoop Data

This is a guest post by Eric Czech, Chief Architect at Next Big Sound, talks about some unique approaches taken to solving scalability challenges in music analytics.

Tracking online activity is hardly a new idea, but doing it for the entire music industry isn't easy. Half a billion music video streams, track downloads, and artist page likes occur each day and measuring all of this activity across platforms such as Spotify, iTunes, YouTube, Facebook, and more, poses some interesting scalability challenges. Next Big Sound collects this type of data from over a hundred sources, standardizes everything, and offers that information to record labels, band managers, and artists through a web-based analytics platform.

While many of our applications use open-source systems like Hadoop, HBase, Cassandra, Mongo, RabbitMQ, and MySQL, our usage is fairly standard, but there is one aspect of what we do that is pretty unique. We collect or receive information from 100+ sources and we struggled early on to find a way to deal with how data from those sources changed over time, and we ultimately decided that we needed a data storage solution that could represent those changes.  Basically, we needed to be able to "version" or "branch" the data from those sources in much the same way that we use revision control (via Git) to control the code that creates it.  We did this by adding a logical layer to a Cloudera distribution and after integrating that layer within Apache Pig, HBase, Hive, and HDFS, we now have a basic version control framework for large amounts of data in Hadoop clusters.

As a sort of "Moneyball for Music," Next Big Sound has grown from a single server LAMP site tracking plays on MySpace (it was cool when we started) for a handful of artists to building industry-wide popularity charts for Billboard and ingesting records of every song streamed on Spotify. The growth rate of data has been close to exponential and early adoption of distributed systems has been crucial in keeping up. With over 100 sources tracked coming from both public and proprietary providers, dealing with the heterogenous nature of music analytics has required some novel solutions that go beyond the features that come for free with modern distributed databases.

Next Big Sound has also transitioned between full cloud providers (Slicehost), hybrid providers (Rackspace), and colocation (Zcolo) all while running with a small engineering staff using nothing but open source systems. There was no shortage of lessons learned in this process and we hope others can take something away from our successes and failures.

How does Next Big Sound make beautiful data out of all that music? Step inside and see...

Categories: Architecture

Simple sketches for diagramming your software architecture

Coding the Architecture - Simon Brown - Tue, 01/28/2014 - 17:24

Are you struggling to communicate/visualise the architecture of your software? If so, regardless of whether your software system is bespoke or based upon a platform (e.g. Microsoft SharePoint, Dynamics CRM, etc), you might be interested in a short article that I wrote for Methods & Tools. It's a summary (with examples) of the "C4" approach that I talk about in my Software Architecture for Developers book.

If you're working in an agile software development team at the moment, take a look around at your environment. Whether it's physical or virtual, there's likely to be a story wall or Kanban board visualising the work yet to be started, in progress and done. Visualising your software development process is a fantastic way to introduce transparency because anybody can see, at a glance, a high-level snapshot of the current progress. As an industry, we've become pretty adept at visualising our software development process over the past few years although it seems we've forgotten how to visualise the actual software that we're building. I'm not just referring to post-project documentation, this also includes communication during the software development process. Agile approaches talk about moving fast, and this requires good communication, but it's surprising that many teams struggle to effectively communicate the design of their software.

You can read "Simple sketches for diagramming your software architecture" on Methods & Tools.

Categories: Architecture

Day 7 of 7 Days of Agile Results–Saturday (Have Fun)

imageYour Outcome: Design a great weekend.

Welcome to Day 7 of 7 Days of Agile Results.  Agile Results is the productivity system introduced in my best-selling time management book, Getting Results the Agile Way.

Let’s recap what we’ve done so far:

Today, it’s all about fun.

Use Agile Results to Make the Most of Your Weekend

What do you want out of this weekend?

You can use The Rule of 3 whenever you need it.    Simply identify 3 Wins or 3 Outcomes that you want to achieve.

So, what are 3 Wins that I want out of this weekend?

  1. Catch up on sleep.
  2. Catch up on movies.
  3. Learn something deep.

I caught up on sleep, and I went to see Long Survivor.   It was intense.   Hopefully, I’ll catch up on a few more movies tomorrow.

I also plan to go through a home study course that I’ve had for a while.  Now’s the time.

3 Takeaways from 7 Days of Agile Results

Hopefully, stepping through each day had helped you get started with Agile Results.

1. If you fall off the horse, you can start Agile Results anytime, anywhere, simply by asking yourself:

“What are 3 outcomes that I want for today?”

… and write your answers down on a piece of paper or sticky note (my tool of choice.)

That’s it, you’re doing Agile Results.

2. If you remember two things, remember to use Three Wins or The Rule of 3:

Three Wins for the Day, Three Wins for the Week, Three Wins for the Month, Three Wins for the Year

3. If you remember three things, then remember the Monday Vision, Daily Outcomes, Friday Reflection pattern:

  1. Monday Vision: On Mondays, identify your 3 outcomes or 3 Wins for the week.
  2. Daily Outcomes:  Each day, identify 3 Wins or 3 outcomes for your day.
  3. Friday Reflection:  Each Friday, identify 3 things going well and 3 things to improve.

If you really embrace and practice this simple system, you’ll be turning intentional action into magnificent results.  You’ll also very quickly multiple your results, simply by getting better at focusing, and concentrating your effort.  And, Friday Reflection will help you learn faster and deeper about your own ability to execute, and will help you build your execution excellence muscle.

Write Your Story Forward and Help 3 Others Unleash Their Potential

Don’t get discouraged if you struggle with some things.  Remember that you get to practice every day.  Enjoy it and challenge yourself.  Remember that the value is really in the change, and that our challenges produce some of our best changes.  Our challenges and changes are also the breeding ground of some of our best stories in our lives. 

It’s in this way that we are the authors of our lives and we continuously write our stories forward.

Also remember that sometimes the best way to learn something is to teach others.

If you teach 3 friends (family, or colleagues) how to get started with Agile Results, you’ll be helping more people find a way to flourish and realize their potential and to unleash the productive artist we all have inside.

Best wishes on your journey ahead.

You Might Also Like
Categories: Architecture, Programming

Stuff The Internet Says On Scalability For January 24th, 2014

Hey, it's HighScalability time:


Gorgeous image from Scientific American's Your Brain by the Numbers
  • Quotable Quotes: 
    • @jezhumble: Google does everything off trunk despite 10k devs across 40 offices. 
    • @KentLangley: "in 2016. When it goes online, the SKA is expected to produce 700 terabytes of data each day" 
    • Jonathan Marks: It's actually a talk about how NOT to be creative. And what he [John Cleese] describes is the way most international broadcasters operated for most of their existence. They were content factories, slave to an artificial transmission schedule. Because they didn't take time to be creative, then ended up sounding like a tape machine. They were run by a computer algorithm. Not a human soul. There was never room for a creative pause. Routine was the solution. And that's creativities biggest enemy.

  • 40% better single-threaded performance in MariaDB. Using perf, cache misses were found and the fix was using the right gcc flags. But the big hairy key idea is: modern high-performance CPUs, it is necessary to do detailed measurements using the built-in performance counters in order to get any kind of understanding of how an application performs and what the bottlenecks are. Forget about looking at the code and counting instructions or cycles as we did in the old days. It no longer works, not even to within an order of magnitude.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge...

Categories: Architecture

Your Invited to the Texas Association of Enterprise Architects January Meeting

Mike Walker's Blog - Fri, 01/24/2014 - 15:08

 Texas AEA

The Texas Association of Enterprise Architects (AEA) has launched!

From the President of the Texas AEA Chapter,

I am very excited to establish this new chapter of the Global Association of Enterprise Architects. It brings with it great prestige and a rich network of over 35,000 professional architects world-wide. We live in a vibrant community of Texas professionals, vendors and consortiums that offers thought leadership, visions for the future, and proven practices based on reality of today. The Texas AEA chapter has the following objectives:

  • create a model for the profession
  • exchange knowledge
  • discuss new and emerging trends
  • establish working groups

We will keep you up to date on the forthcoming website that will be up shortly where you can download all the latest happenings.

Best Regards,

Mike Walker

Events

Below are some upcoming events where you can learn more about the organization and network with your peers. We look forward to seeing you at one of our upcoming events.

 

January 30 – 1st Monthly Meeting

 

Join us for our very first monthly meeting. The meeting will take place on Thursday, January 30, 5:30 – 7:30 at Iron Cactus North Austin on Stonelake Blvd, Austin, TX.  (see new venue below).

This meeting will be a great opportunity for you to get plugged into the local architecture community along with an introduction to the Texas AEA Chapter.

To join RSVP here.

 

[Update! New Venue Selected]

I have an important announcement for all registered attendees for our Texas AEA meeting has been so popular that we have exceeded the capacity of our last meeting location! Thank you for your interest in the Texas AEA. Given this situation we needed a larger location that support us and we found a perfect one for us.

The new location information can be found below:

clip_image001

Able’s North Austin

4001 West Parmer Lane

Austin, Texas 78727

(512) 835-0010

Note: Able’s is behind other buildings and you should look for It’s a Grind coffee shop and turn into that small shopping plaza to find Able’s.

 

February 23 – Monthly Meeting

Monthly Meeting: Architecture topics and networking social. Details coming soon.

 

March 20 – Texas AEA Conference

We are pleased to announce that the Texas Architecture Conference will be held in Austin, Texas.

 

A thank you to our sponsors!

 Texas AEA Sponsors

Categories: Architecture

Day 6 of 7 Days of Agile Results – Friday (Friday Reflection)

imageYour Outcome: Identify 3 things going well and 3 things to improve based on you results for this week.

Welcome to Day 6 of 7 Days of Agile Results.  Agile Results is the productivity system introduced in my best-selling time management book, Getting Results the Agile Way.

Let’s recap what we’ve done so far:

Use Friday Reflection as a way to invest in yourself, reinvent yourself, and renew yourself.   If you do this well, this is the secret of continuous improvement.  Each week, what you learn from Friday Reflection can help you tune and improve your results so you get better and better.

One of the most common patterns is to simply lose sight of what we set out to achieve for his week.  That's why thinking of 3 Wins for the Week is so powerful.  It gives us a target.  We check ourselves during the week, and adjust our course.  Then Friday is where we really peer into our personal process and find ways to improve it.

3 Things Going Well and 3 Things to Improve

To do Friday reflection, simply give yourself 10 or 20 minutes on Friday mornings to ask yourself two things:

  1. What are three things going well?
  2. What are three things to improve?

The goal is to carry the good forward and build better habits.

Before you answer the questions above, really reflect on your week.  Did you do what you set out to do?  If not, did you trade up for the right things?  Did you get randomized? Did you bite off more than you can chew?

See what starts to happen?  You start to notice your own patterns.  This awareness becomes your advantage, when you use it to change what's not working, and do more of what is working  It's a way to improve your personal habits and streamline your results.

This is a powerful way to learn your own capacity and to gradually improve your “ability to execute.”

Example Patterns

Here are a few common patterns and what to do about them:

  1. You completed none of what you set out to do for the week.   Pay a lot of attention to this.   What’s important is that you know “WHY” you didn’t complete what you set out to do.  If you traded up for higher value impact, congratulations.   If, you simply got randomized, and distracted, then notice how much better today would feel if you had stayed focused on your 3 outcomes.  Also, make it a point to identify how you choose better outcomes for the week.   You want to really start to learn what sorts of things are high value results for you.
  2. You completed some of what you set out to do.  Congratulations.   At least that’s some progress you can be proud of.   Use your results as feedback.   Did you complete your best opportunities?   Or did you simply go for low-hanging fruit?   Did you try to accomplish more than was reasonable within this time frame?   Use your answers here to gain tremendous insight into your own bottlenecks and ability to execute.   If you are simply biting off too much, then try to improve your ability to estimate what you can achieve within a week.   You’ll get better at this each time.  You’ll start to be able to size up your work week simply by eye-balling it.   You’ll develop an intuitive ability to guestimate your efforts, which will help you right size your wins, and focus on the highest impact results.   This, in turn, will help you build momentum so that you can create a snowball of progressive impact.
  3. You completed what you set out to do, but it doesn’t feel like impact.    This is a good learning, as well as a good reminder.   First, make sure you aren’t throwing your results away.   Check to see if you are giving yourself enough credit for achieving your results.   Practice an attitude of gratitude.   Next, see if you can put your finger on “WHY” it doesn’t feel like impact.   Did you simply have  lack of clarity around what would be significant for this week?   Did you simply set your bar too low?   The best insight here is to gain increasing clarity on what types of outcomes really constitute high value results.
Enjoy the Journey

Most important, relax and truly embrace Friday Reflection as one of the best ways that you can improve your personal performance, in a simple and almost automatic way.   Just by asking these simple questions, you start to gain awareness and you start to gain more clarity on what works best for you, and what your true execution abilities are.

What’s even better is that you’ll naturally start to improve here simply by paying attention to these key questions.   They’ll help you see what you did not see before.  And, if you make this Friday Reflection a simple way to check in with yourself each week, you’ll find yourself paying more attention throughout the week.  And, you’ll start to make little adjustments here and there that start to help you focus and prioritize on higher value outcomes and activities

You will gradually find that you are achieving better and better results, with less effort and more clarity.

You will unleash the productive artist that’s already within you, as well as tap your most inspiring abilities, skills, and strengths.

This is how you will fan your flames and set your productivity on fire … the quiet way … the Agile way.

Welcome to your private victories that last a life time.

You Might Also Like
Categories: Architecture, Programming

Day 5 of 7 Days of Agile Results – Thursday (Daily Outcomes)

imageYour Outcome: Know your 3 Wins to target today so that you have a simple way to focus and prioritize your effort throughout your day.

Welcome to Day 5 of 7 Days of Agile Results.  Agile Results is the productivity system introduced in my best-selling time management book, Getting Results the Agile Way.

Let’s recap what we’ve done so far:

Hopefully, at this point, it’s getting easier for you to identify your 3 outcomes for the day.

If not, fear not. 

You will get better, as long as you make it a conscious effort to really figure out what 3 outcomes you want for today.

Picture Your Day, the Agile Way

Hopefully, you’re also benefiting from scanning your calendar to get a good mental picture of your day.  And by doing a quick list of your main tasks for the day, this should also help you practice prioritizing before just diving in.  

You’re teaching yourself to focus on higher value activities.

You can actually use this simple picture of your day to help envision your results and inspire you. 

All you have to do is imagine a simple future scene or two for your morning, a simple future scene or two for your afternoon, and a simple future scene or two for your night. 

And, if you don’t like what you imagine, then re-imagine it, and play out new possibilities.

A Busy Day, But There’s Always a Chance for Bursts of Brilliance

For today, I have an exceptionally busy calendar, but I notice that I have a few time slices where I can really focus and nail a few things.

My 3 planned outcomes for today are:

  1. I have the Devices and Services Story slides 80% complete (and with the right “fingerprints” on them) so that I can shop it around.
  2. I have a test scenario to walk the Modern App Transformation Story so that I can validate our approach before sinking a bunch of time.
  3. I have a simple plan for landing and scaling the Devices and Services readiness material across the organization so I can start to put the right things in the right places.

If I accomplish those three things, I’ll really be in great shape.   I have a lot of “below the line” things to do today, and a schedule that will randomize me quite a bit, but if I play my cards right, I should be able to pull off my 3 Wins above.

Play Around with How You State Your Outcomes or Wins

One thing I’ll point out, and which I hope you noticed, is that each day, I’m playing around with how I represent the 3 Wins (or 3 Outcomes, or 3 Results.)   I want you to play around to so that you find what works for you.   I’m simply showing you a few different variations so that you can see that it’s not about doing it this way or that way that’s important.  

What’s important is that you have clarity on the 3 things you want out of today and for the week.   I can’t emphasize how important that is as a tool to help you focus and prioritize.

Appreciate Your Results

But what it also helps you do is to appreciate what you accomplish.  It gives you a simple way to play back your results, and acknowledge your achievements.  

It sounds so simple, and it is, but when you appreciate your results, you breathe new life into all your efforts, and you build momentum like you wouldn’t believe.

Don’t throw that away.  

Your results are actually your own reward.  

Cherish your achievements, and enjoy the journey as you go.

Have a great day, the agile way.

You Might Also Like
Categories: Architecture, Programming

How would you build the next Internet? Loons, Drones, Copters, Satellites, or Something Else?

If you were going to design a next generation Internet at the physical layer that routes around the current Internet, what would it look like? What should it do? How should it work? Who should own it? How should it be paid for? How would you access it?

It has long been said the Internet routes around obstacles. Snowden has revealed some major obstacles. The beauty of the current current app and web system is the physical network doesn't matter. We can just replace it with something else. Something that doesn't flow through choke points like backhaul networks, under sea cables, and cell towers. What might that something else look like?

Google's Loon Project

Project Loon was so named because the idea was thought to be loony. Maybe not...

Categories: Architecture

Day 4 of 7 Days of Agile Results - Wednesday (Daily Outcomes)

Your Outcome: Celebrate your personal victories for the week so far and set yourself up for success for today, and for the rest of the week.

Happy Hump Day!

Today is a great day to checkpoint your progress against your 3 outcomes for the week, and re-adjust if necessary.

But first, let's recap where we are so far:

What we’re doing to do now is check how we’re doing against the week.  This may impact what we choose to focus on today.

For example, here is the list my 3 Wins for this week that I want to accomplish:

  1. Dry run of the Devices and Services training deck.
  2. Actionable feedback and next steps based on a walkthrough of all the Enterprise Identity assets.
  3. A test case to use for the Modern App Transformation story to test initial thinking.

I’ve already achieved the first two outcomes.  As far as #3, it’s starting to feel more like a stretch goal.   I also underestimated how much further I need to take #1, as additional requirements start to come in.  But the more important thing I note is that my backlog of administrative activities has really ballooned out of control to the point where I need to put a solid dent in it today.

The Agile part here is about always learning, adapting, and improving based on what you learn, and what you want to achieve within the time you and energy you have available.

Next, I take a look at my calendar, and I jot down a few of the key items that stand out.  I have a presentation first thing in the morning, followed by a few key meetings, and a few sprinkled in the afternoon.   I notice that I have a few open blocks of time where I can focus on some of my heavy lifting.

With that in mind, let me list out some of the key activities that I have for today:

- Present my summary and highlights of the Intellectual Property development effort for the past 1/2 year, and the plan for the next six months
- Review the budget plan and projected spend
- Create additional slides for the Devices and Services story
- Help one of my mentees with their career planning and next immediate moves
- Meet with leads from each of the partner teams to validate our initial plan
- Set up a series of meetings for next week with colleagues in town from around the world
- Get detailed feedback from the first wave of results from the off-shore development team
- Create a plan for scaling the delivery and self-paced training across the organization
- Hunt down a few colleagues that I need help on a few technical details and drill-downs

I have a lot to do, but it’s not about quantity.  It’s about impact.  And, I need a simple set of “tests for success for today.”

Here’s what I choose as my 3 Wins for today to help me stay on track:

  1. The team feels good about the results of the Intellectual Property development efforts so far and believes in the vision
  2. The Devices + Services story includes all the latest feedback (even if just in raw form.)
  3. My administrative backlog is chopped down to size.

My presentation is already scheduled, so all I’m going to do is show up with a simple message that I want people to take away.   I have a few stories to share, and if people want details, I have more than enough, so I’m set there.

What I need to do to pull off my other two wins is dedicate a few of my open time blocks in the afternoon to really focus on them.  To nail my administrative backlog, I’m going to do as many things in a batch as possible, while I have my “administrative hat” on.

To nail the Devices + Services story and integrate the feedback, I’ll need to use some of the time slices earlier in my day to do some initial preparation.      I’ll also need to set expectations with my team that I’ll be heads down while I work through it.

If I achieve these 3 Wins, I’ll have significant progress, and I’ll be on track for the most important things.  I still have a lot of heavy lifting, but at least I have clarity of what my most important outcomes are.

With this in mind, I try to get a clear and simple picture of my day.   My morning will be about presenting and attending meetings.  I’ll use lunch as a chance to play out some ideas on the Devices + Services story.  Then, my afternoon will be about turning ideas into slides, and nailing my administrative backlog. 

I can see it min mind.  And, as the saying goes, “What the mind can conceive, the body can achieve.”  But mostly I like the peace of mind that comes from having clarity on my priorities and having a simple picture of my day that I can use a baseline.

Will my day get messy?  You bet.  In fact, I expect the unexpected.  But will I be able to roll with the punches? Yes, a lot better now that I have a map of my day, and I know that’s truly important.  Now, I can deliberately trade up for new opportunities, or respond to challenges more effectively as they come my way.

Today is a big hump in the week.  

But, through some quick and simple planning and map, I’m already on top of it.

You Might Also Like

Day 1 of 7 Days of Agile Results - Sunday (Getting Started)

Day 2 of 7 Days of Agile Results – Monday (Monday Vision)

Day 3 of 7 Days of Agile Results – Tuesday (Daily Outcomes)

Categories: Architecture, Programming

Sponsored Post: Netflix, Logentries, Host Color, Booking, Apple, MongoDB, BlueStripe, AiScaler, Aerospike, LogicMonitor, AppDynamics, ManageEngine, Site24x7

Who's Hiring?
  • Apple is hiring for multiple positions. Imagine what you could do here. At Apple, great ideas have a way of becoming great products, services, and customer experiences very quickly.
    • Sr. Security Software Developer. We are looking for an excellent programmer who's done extensive security programming. This individual will participate in various security projects from the start to the end. In addition to security concepts, it's important to have intricate knowledge of different flavors of Unix operating systems to develop code that's compact and optimal. Familiarity with key exchange protocols, authentication protocols and crypto analysis is a plus. Please apply here.
    • Senior Server Side Engineer. The Emerging Technology team is looking for a highly motivated, detail-oriented, energetic individual with experience in a variety of big data technologies.  You will be part of a fast growing, cohesive team with many exciting responsibilities related to Big Data, including: Develop scalable, robust systems that will gather, process, store large amount of data Define/develop Big Data technologies for Apple internal and customer facing applications. Please apply here.
    • Senior Server Side Engineer. The Emerging Technology team is looking for a highly motivated, detail-oriented, energetic individual with experience in a variety of big data technologies.  You will be part of a fast growing, cohesive team with many exciting responsibilities related to Big Data, including: Develop scalable, robust systems that will gather, process, store large amount of data Define/develop Big Data technologies for Apple internal and customer facing applications. Please apply here.
    • Senior Engineer: Emerging Technology. Apple’s Emerging Technology group is looking for a senior engineer passionate about exploring emerging technologies to create paradigm shifting cloud based solutions. Please apply here
    • Sr Software Engineer. The Emerging Technology team is looking for a highly motivated, detail-oriented, energetic individual with experience in a variety of big data technologies. You will be part of a fast growing, cohesive team with many exciting responsibilities related to Big Data. Please apply here.

  • The Netflix Cloud Performance Team is hiring. Help tackle the more complex scalability challenges emerging on the cloud today, wrangling tens of thousands of instances, handling billions of requests a day. We are searching for a Senior Performance Architect and a Senior Cloud Performance Tools Engineer

  • We need awesome people @ Booking.com - We want YOU! Come design next generation interfaces, solve critical scalability problems, and hack on one of the largest Perl codebases. Apply: http://www.booking.com/jobs.en-us.html

  • UI EngineerAppDynamics, founded in 2008 and lead by proven innovators, is looking for a passionate UI Engineer to design, architect, and develop our their user interface using the latest web and mobile technologies. Make the impossible possible and the hard easy. Apply here.

  • Software Engineer - Infrastructure & Big DataAppDynamics, leader in next generation solutions for managing modern, distributed, and extremely complex applications residing in both the cloud and the data center, is looking for a Software Engineers (All-Levels) to design and develop scalable software written in Java and MySQL for backend component of software that manages application architectures. Apply here.
Fun and Informative Events
  • Your amazing event here.
Cool Products and Services
  • Log management made easy with Logentries Billions of log events analyzed every day to unlock insights from the log data the matters to you. Simply powerful search, tagging, alerts, live tail and more for all of your log data. Automated AWS log collection and analytics, including CloudWatch events. 

  • HostColorCloud Servers based on OpenNebula Cloud automation IaaS. Cloud Start features: 256 MB RAM; 1000 GB Premium Bandwidth; 10 GB fast, RAID 10 protected Storage; 1 CPU Core;  /30 IPv4 (4 IPs) and /48 IPv6 subnet and costs only $9.95/mo. Client could choose an OS (CentOS is default). 

  • LogicMonitor is the cloud-based IT performance monitoring solution that enables companies to easily and cost-effectively monitor their entire IT infrastructure stack – storage, servers, networks, applications, virtualization, and websites – from the cloud. No firewall changes needed - start monitoring in only 15 minutes utilizing customized dashboards, trending graphs & alerting.

  • MongoDB Backup Free Usage Tier Announced. We're pleased to introduce the free usage tier to MongoDB Management Service (MMS). MMS Backup provides point-in-time recovery for replica sets and consistent snapshots for sharded systems with minimal performance impact. Start backing up today at mms.mongodb.com.

  • BlueStripe FactFinder Express is the ultimate tool for server monitoring and solving performance problems. Monitor URL response times and see if the problem is the application, a back-end call, a disk, or OS resources.

  • Intel-Aerospike AppNexus Case Study. This new Intel case study explains how the world's largest independent advertising platform, AppNexus, increased their performance, reliability and serviceability while reducing each cluster size from about 50 machines to 8 by integrating Intel SSDs in a wide SATA configuration with the Aerospike NoSQL database. Download the case study to learn more.

  • aiScaler, aiProtect, aiMobile Application Delivery Controller with integrated Dynamic Site Acceleration, Denial of Service Protection and Mobile Content Management. Cloud deployable. Free instant trial, no sign-up required.  http://aiscaler.com/

  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.

  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below. Please click to read more...

Categories: Architecture

Day 3 of 7 Days of Agile Results – Tuesday (Daily Outcomes)

imageYour Outcome:  Learn how to use Daily Outcomes to identify 3 outcomes or 3 Wins for today.  By identifying your best 3 Wins for the day, you’ll be able to focus and prioritize throughout the day to achieve better results.

Welcome to Day 3 of 7 Days of Agile Results.  Agile Results is the productivity system introduced in my best-selling time management book, Getting Results the Agile Way.

Just to do a quick recap, here’s what we’ve done so far:

Now, for today, let’s get started.

It’s a fresh start.  This is your chance to choose the best things to focus on that will help you make the most impact today.

Here’s a simple process you can use to get started:

  1. Scan your calendar so you can get a good picture of the key events in your day.  You want to get a good sense of the priorities.
  2. Write down a simple list of the key tasks you have on your plate for today.
  3. Now, at the top of your list, identify 3 outcomes that would make this a great day.   Think of these as your 3 Wins for today, to help you focus and prioritize throughout your day.

For example, here are my 3 outcomes that I want for today:

  1. People in the meeting buy into the Scenarios + Architecture + Value approach
  2. Review meeting of the Devices + Services story leads to closure of open issues
  3. Sync up leads to a breakthrough I can apply to our production process

Those then act as my “tests for success” for the day.  Do I have a lot of tasks on my plate for the day?  You bet.

Do I have a lot of meetings to attend?  Yep.

Will I be trying to use some of the little time slices in my day to try and complete many of my tasks?  Of course.

Will I be dealing with interruptions throughout the day, as well?  Yes, to that, too.

I will be dealing with chaos while riding the dragon.  And throughout the day, I’ll be driving to my 3 outcomes.

They are my North Star, while I deal with whatever comes my way throughout the day.

May your 3 Wins guide you and provide you with clarity, conviction, and calmness among the chaos – TODAY.

You Might Also Like

Day 1 of 7 Days of Agile Results - Sunday (Getting Started)

Day 2 of 7 Days of Agile Results – Monday (Monday Vision)

10 Big Ideas from Getting Results the Agile Way

Agile Results on a Page

The Values of  Agile Results

Categories: Architecture, Programming

8 Ways Stardog Made its Database Insanely Scalable

Stardog makes a commercial graph database that is a great example of what can be accomplished with a scale-up strategy on BigIron. In a recent article StarDog described how they made their new 2.1 release insanely scalable, improving query scalability by about 3 orders of magnitude and it can now handle 50 billion triples on a $10,000 server with 32 cores and 256 GB RAM. It can also load 20B datasets at 300,000 triples per second. 

What did they do that you can also do?

Categories: Architecture

Day 2 of 7 Days of Agile Results - Monday (Monday Vision)

imageYour Outcome:  Learn how to use Monday Vision to identify 3 outcomes or 3 Wins for the week.  By identifying your best 3 Wins for the week, you’ll be able to focus and prioritize throughout the week to achieve better results.

Welcome to Day 2 of 7 Days of Agile Results.  Agile Results is the productivity system introduced in my best-selling time management book, Getting Results the Agile Way.

Today’s focus is on identifying your 3 outcomes for the week.

Have you ever had a week fly by and when Friday hits you’re asking yourself, “Where did the week go?” 

Not any more. 

With Monday Vision, you take a moment to jump ahead to the end of the week, and you ask yourself, “If this were Friday, what are three outcomes, wins, or results that I would want under my belt?” 

Monday is your chance to create a compelling vision for your week. 

It’s flexible, but I’m going to walk you through a specific approach, so that you know how to start.  When you first start, it might take you a little longer, but with practice, it really is just a 10 minute exercise, if that.

Save More Time and Effort While Creating More Value This Week

You’re going to spend a week of your time.   Slow down to speed up.  Yeah, it really works.  This is a quick and simple way to get more value out of the time you already spend, and you can save a lot of time each day and each week by focusing on the key things that matter the most.  If you can shave off things that matter less, to focus on things that matter more, you win the game

There will always be more to do than you can effectively do, so this is a way to set a healthy bar for results.

The summary of what you’re going to do today is to write out a quick list of the things on your plate for this week, and then identify the 3 most important outcomes for the week.   Next, you’ll list out your tasks for today, and then identify your 3 most important outcomes for today.

Let’s go to it  …

Step 1 – Write Down Your List of Important Things for This week

Write down a quick list of things you want to accomplish this week.   If you need to, scan your calendar for the week to remind yourself of some of the big activities you have, key meetings, and any significant events.

Here is a simple example of a dump of actions and things going on for the week:

Example List of Tasks for the Week

- Create a draft of the slides for the Devices and Services story.
- Map out the big rocks for work for Q3 and Q4.
- Review the scenarios, architecture, and value story for Enterprise Identity story.
- Create a simple plan for integrating sales collateral with delivery assets.
- Schedule a working session to figure out the Modern App Transformation approach.
- Meet with Steve to collaborate on how to integrate our methods, models, and approach for helping Enterprise customers with Devices and Services.
- Meet with field leads to get detailed feedback on the information product model and any adoption blockers or issues.
- Team meeting with field to walkthrough new releases and latest evolution of the framework.

There’s more I could add, but for now, let’s use above as a sample working set.  And, obviously there are a bunch of sub-activities and tasks to actually complete each item above.

The idea is to quickly generate a simple list of things you'll be working on this week.   By having this list at your fingertips, you’ll have a pretty good idea of what this week is about, and where you need to spend most of your time and energy.

Step 2 – Write Down 3 Wins that You Want to Accomplish for This Week

Now, at the top of your list, write down 3 things that you really want to accomplish for this week.   These will be the goals that remind you what’s important throughout the week.

Here is an example of the same list, but now I’ve written down my 3 most important goals and achievements at the top:

Example List of Tasks for the Week with 3 Wins at the Top

  1. Devices + Services slides are ready to go.
  2. The Enterprise Identity story is ready to ship.
  3. FY15 plans are in place (straw man)

- Create a draft of the slides for the Devices and Services story.
- Map out the big rocks for work for Q3 and Q4.
- Review the scenarios, architecture, and value story for Enterprise Identity story.
- Create a simple plan for integrating sales collateral with delivery assets.
- Schedule a working session to figure out the Modern App Transformation approach.
- Meet with Steve to collaborate on how to integrate our methods, models, and approach for helping Enterprise customers with Devices and Services.
- Meet with field leads to get detailed feedback on the information product model and any adoption blockers or issues.

I still have a long list of things to do (it’s longer than I’m showing here), but I have listed my 3 most important outcomes at the top.  Those are my tests for success.  If I can accomplish those, then I should feel good about my progress and achievements.   The assumption here is that I did a good job of picking the 3 outcomes that deliver the most value, and that I have the ability to execute

If I didn’t, I will learn and adapt during the week, and especially when I get to Friday Reflection this Friday.

Chances are you have more to do than you have time to do, so this is really an exercise in prioritization.  But it’s also an exercise in simplification.  You could easily generate a laundry list of everything on your plate, but that’s not the idea.  The goal is to get clear on 3 things that you can achieve this week, that would make your best impact and help you focus and prioritize throughout the week.

It’s about getting pragmatic about what you can actually achieve this week, and what actually matters.   There’s no point in focusing on a bunch of things you won’t do.

Step 3 – Write Down 3 Wins that you Want to Accomplish Today

Since this is Monday, not only do I want to identify my 3 Wins for the week, I also want to identify my 3 Wins or 3 Outcomes for today.   They may or may not directly support my 3 Wins for the week, but they at least help me achieve what I need to accomplish today.

Keep in mind, that since I now know my 3 most important outcomes for the week, I can choose better outcomes today to support my week.  Awareness actually helps us avoid letting the week fly by and then wondering where it went and why we didn’t achieve the things that matter.

Similar to the week, I’ll list out my activities for today, but it’s a much faster process, since I can glance at my calendar for the day, and I already have several things that are top of mind for today.   But, the key step here is to identify my 3 Wins for today.

For example, for today, my 3 Wins that I want to accomplish are:

Example List of 3 Outcomes for Today

  1. Dry run of the Devices and Services training deck.
  2. Actionable feedback and next steps based on a walkthrough of all the Enterprise Identity assets.
  3. A test case to use for the Modern App Transformation story to test initial thinking.

Each of those outcomes is a valuable results and paves the way for other work that will help me have a high-impact week.

Do I have many, many other things to also accomplish?  You bet.   But if I achieve those 3 goals, then I will have accomplished my most important results.

It’s also worth pointing out that each of those goals has a lot of steps and sub-steps and activities within them.  The point is that I’m not tracking tasks.  I’m focusing on outcomes. and I let my brain do what it does best, which is figure out the right tasks as I go, as I’m working towards each outcome.   But the most important things to keep front and center, are the outcomes to work towards.   Otherwise, it’s easy to get lost among a sea of activities, and a jungle of tasks, and lose sight of the priorities and outcomes that matter.

Side note: I do keep a list of the tasks I’m working on, but I focus on the outcomes.  A simple thing to keep in mind is write your To Do list for today, and then add your 3 Wins or 3 Outcomes to the top.

Recap

So, to recap, here’s what we did:

  1. We listed out our work for the week, and then wrote down our 3 outcomes that we want to achieve for the week.  This is Monday Vision.
  2. We listed out our work for the day, and then wrote down our 3 outcomes for today.  This is Daily Wins.
  3. We produced a simple list of 3 Wins for this week and 3 Wins for today that we will try to achieve.  This helps us start Monday off with a clear vision of impact for the week, and for today, in a very simple way.

One thing to keep in mind is that it’s not that you are limiting yourself to “3 things.”  It’s achieving 3 Wins.  You are focusing on achieving 3 significant outcomes that matter each day, and each week.   Think of it as a way of turning a chaos of activity into simple goals that are easy to remember and pull everything together into something that matters.

Assignment
  1. Review your 3 Wins for this week.   Are they simple and sticky to say out loud?  Do they really represent value?  If this week were over, are those really the 3 things that you would regret if you didn’t accomplish them?
  2. Review your 3 Wins for today.  If you accomplished those by the end of today, would you feel like you had a successful day?  If not, then go back to the drawing board and identify 3 outcomes that you can both realistically achieve and that you will feel good about.

It’s important to remind yourself that you can’t do everything, and that you especially can’t do everything all at once.  So, the best you can do is spend the right time, on the right things, with the right energy, the right way.   If you keep this little formula in mind, and apply it when you choose your 3 outcomes to focus on for today, you will be ahead of the game.

In fact, that’s also a great way to help you choose your 3 outcomes for today.  Ask yourself, “What are the 3 best outcomes I can produce today with the time and energy I have available?”

Congrats for completing day 2 of 7 Days of Agile Results.

And note, if you’ve successfully identified 3 compelling Wins for today and 3 compelling Wins for the week, you are ahead of 99% of the population who has not and will not take the time to actually do this.    You’ve made your day more meaningful, your week more meaningful, and you’ve created a way to better prioritize and focus throughout the week.

You have set a powerful stage for your success throughout the week.

You Might Also Like

7 Days of Agile Results: Day 1 - Sunday (Getting Started)

10 Big Ideas from Getting Results the Agile Way

Agile Results on a Page

How I Use Agile Results

The Values of  Agile Results

Categories: Architecture, Programming

Day 1 of 7 Days of Agile Results - Sunday (Getting Started)

imageYour Outcome:  Learn how to get started quickly with Agile Results, even if you've never read the book.

Welcome to day 1 of 7 Days of Agile Results.  By following along each day, you can learn how to use Agile Results as a powerful system to easily triple your productivity.

Agile Results is the productivity system introduced in my best-selling time management book, Getting Results the Agile Way.

You don't have to invest a lot of time, to get a lot of benefits.  You don't even need any ramp up time.  You can just get started now and implement other concepts from the book as time goes by, and when you are ready for those concepts.

The main idea is to get benefits now vs. later.

Here are some of the big benefits you'll gain from using Agile Results as your productivity system:

  1. You'll automatically master time management, motivation, and personal productivity by using proven practices.
  2. You'll improve and amplify your impact by focusing on your top 3 Wins for the week.
  3. You'll avoid getting overloaded and overwhelmed by your day by using 3 Wins each day to focus and prioritize your time, energy, and attention.
  4. You'll reduce your stress and improve your peace of mind by generating a vision for your week and for each day.
  5. You'll improve your satisfaction each day by knowing that you actually got the important things done.
  6. You'll quickly learn how to improve your productivity through continuous learning, feedback, and changing your approach.
  7. You'll learn how to embrace change and get better results in any situation.

With that in mind, let's take a quick look at how Agile Results works:

  1. On Mondays, write down 3 outcomes or "Wins" that you want for the week. (This is called Monday Vision.)
  2. Each day, write down 3 outcomes or "Wins" that you want for each day. (This is called Daily Outcomes.)
  3. On Fridays, ask yourself, "What are 3 things going well?," and, "What are 3 things to improve?", and write your answers down. (This is called Friday Reflection.)

This is the Monday Vision, Daily Wins, Friday Reflection pattern in Agile Results.  It’s a simple pattern for weekly results.  By focusing on 3 Wins each week, and each day, you apply concentrated effort to your most important results.  Focus is your force multiplier.

It might sound incredibly simple, but that's the idea.   Agile Results really is a simple system for meaningful results

Simple, but powerful. 

When you identify 3 outcomes or "Wins", what you are doing is:

  1. Prioritizing among your multiple, competing priorities.
  2. Focusing on your highest value results.
  3. Creating a way to more consistently produce better and better results.

What you are actually doing is learning how to identify and produce continuous value … the agile way.

And, as you will learn, value is the ultimate short-cut.

Assignment

  1. Read a few case studies and testimonials to understand some of the results that people achieve using Agile Results.
  2. Add a reminder to your calendar to test-drive Agile Results each day for this week.
Categories: Architecture, Programming

Stuff The Internet Says On Scalability For January 17th, 2014

Hey, it's HighScalability time:


From the stunning Scale of the Universe - Interactive Flash Animation
  • $7 trillion: US spend on patrolling oil sea-lanes; 82 billion: files served by MaxCDN in 5 months
  • Quotable Quotes: 
    • @StephenFleming: "Money doesn’t solve scaling problems, but the actual solutions to scaling problems always cost money." http://daringfireball.net/2014/01/googles_acquisition_of_nest
    • David Rosenthal: Robert Puttnam in Making Democracy Work and Bowling Alone has shown the vast difference in economic success between high-trust and low-trust societies.
    • @kylefox: That's a huge advantage of SaaS businesses: you can be liberal with refunds & goodwill credits w/o impacting the bottom line much.
    • Thomas B. Roberts: That’s the essence of science: Ask the impertinent question, and you are on your way to pertinent science.
    • Benjamin K. Bergen: Simulation is an iceberg. By consciously reflecting, as you just have been doing, you can see the tip—the intentional, conscious imagery. But many of the same brain processes are engaged, invisibly and unbeknownst to you, beneath the surface during much of your waking and sleeping life. Simulation is the creation of mental experiences of perception and action in the absence of their external manifestation.

  • Urbane apps are the future. 80% world population will be in cities by 2045

  • Knossos: Redis and linearizability. Kyle Kingsbury delivers an amazingly indepth model based analysis of "a hypothetical linearizable system built on top of Redis WAIT and a strong coordinator." The lesson: don't get Kyle mad.

  • If a dead startup had a spirit, this is what it would look like: About Everpix. A truly fine memorial. 

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge...

Categories: Architecture

Windows Azure: Staging Publishing Support for Web Sites, Monitoring Improvements, Hyper-V Recovery Manager GA, and PCI Compliance

ScottGu's Blog - Scott Guthrie - Thu, 01/16/2014 - 20:53

This morning we released another great set of enhancements to Windows Azure.  Today’s new capabilities and announcements include:

  • Web Sites: Staged Publishing Support and Always On Support
  • Monitoring Improvements: Web Sites + SQL Database Alerts
  • Hyper-V Recovery Manager: General Availability Release
  • Mobile Services: Support for SenchaTouch
  • PCI Compliance: Windows Azure Now Validated for PCI DSS Compliance

All of these improvements are now available to use immediately (note that some features are still in preview).  Below are more details about them:

Web Sites: Staged Publishing Support

With today’s release, you can now enable staged publishing to your Windows Azure Web Sites.  This new feature is really powerful, and enables you to deploy updates of your web apps/sites to a staging version of the site that can be accessed via a URL that is different from your main site.  You can use this staged site to test your site/app deployment and then, when ready, instantaneously swap the content and configuration between the live site and the staging version. 

This new features enables you to deploy changes with more confidence.  And it ensures that your site is never in an inconsistent state (where some files have been updated and others not) - now you can immediately swap all changes to all of the files in one shot.

Enabling Staged Publishing Support

To setup staged publishing go to the DASHBOARD tab of a web site and click Enable staged publishing from the quick glance section:

image

Clicking this link will cause Azure to create a new staging version of the web-site and link it to the existing site.  This linkage is represented in the navigation of the Windows Azure Management Portal – the staging site will show up as a sub-node of the primary site:

image

If you look closely at the name of the staging site, you’ll notice that its URL by default is sitename-staging (e.g. if the primary site name was “scottgu”, the staging site would be “scottgu-staging”):

image

You can optionally map any custom DNS name you want to the staging site (using either a C-Name or A-Record) – just like you would a normal site.  So your staging domain doesn’t have to have an azurewebsites.net extension.  In the scenario above I could remap the staging domain to be staging.scottgu.com, or scottgu-staging.com, or even foobar.scottgu.com if I wanted to. 

The staging URL doesn’t change between deployments of a site – so you can configure a custom DNS once, and then you can use it across all subsequent deployments.  You can also optionally enable SSL on the staging site and upload a SSL certificate to use with the staging domain (ensuring you can fully test/validate your SSL scenarios before swapping live).

Configuring the Staging Site

You can click on the staging site to manage it just like any other site:

image 

The SCALE tab and the LINKED RESOURCES tabs are disabled for staging sites, but all other tabs work as expected.  You can use the CONFIGURE tab to set configuration settings like database and application connection-strings (if you set these at the site level they override anything you might have in a web.config file).

One thing you’ll also notice when you open the staging site is that there is a new SWAP button in the bottom command-bar of it – we’ll talk about how to use that in a little bit.

Deploying to the Staging Site

Deploying a new instance of your web-app/site to the staging site is really easy.  Simply deploy to it just like you would any normal site.  You can use FTP, the built-in “Publish” dialog inside Visual Studio, Web Deploy or Git, TFS, VS Online, GitHub, BitBucket, DropBox or any of the other deployment mechanism we already support.  You configure these just like you would a normal site.

Below I’m going to use the built-in VS publish wizard to publish a new version of the site to the staging site:

image

Once this new version of the app is deployed to the staging site we can access a page in it using the staging domain (in this case http://scottgu-staging):

image

Note that the new version of the site we deployed is only in the staging site.  This means that if we hit the primary site domain (in this case http://scottgu) we wouldn’t see this new “V2” update - it would instead show any older version that had been previously deployed:

image

This allows us to do final testing and validation of the staging version without impacting users visiting the live production site.

Swapping Deployments

At some point we’ll be ready to roll our staged version to be the live production site version.  Doing this is easy – all we need to do is push the SWAP button within the command-bar of either our live site or staging site using the Windows Azure Portal (you can also automate this from the command-line or via a REST call):

image

When we push the SWAP button we’ll be prompted with a confirmation dialog explaining what is about to happen:

image

If we confirm we want to proceed with the swap, Azure will immediately swap the content of the live site (in this case http://scottgu) with the newer content in the staging site (in this case http://scottgu-staging).  This will take place immediately – and ensure that all of the files are swapped in a single shot (so that you never have mix-matched files).

Some settings from the staged version will automatically copy to the production version – including things like connection string overrides, handler mappings, and other settings you might have configured.  Other settings like the DNS endpoints, SSL bindings, etc will not change (ensuring that you don’t need to worry about SSL certs used for the staging domain overriding the production URL cert, etc).

Once the swap is complete (the command takes only a few seconds to execute), you’ll find that the content that was previously in the staging site is now in the live production site:

image

And the content that had been in the older live version of the site is now in the staging site.  Having the older content available in the staging site is useful – as it allows you to quickly swap it back to the previous site if you discover an issue with the version that you just deployed (just click the SWAP button again to do this).  Once you are sure the new version is fine you can just overwrite the staging site again with V3 of your app and repeat the process again.

Deployment with Confidence

We think you’ll find that the new staged publishing feature is both easy to use and very powerful, and enables you to handle deployments of your sites with an industrial strength workflow.

Web Sites: Always On Support

One of the other useful Web Site features that we are introducing today is a feature we call “Always On”.  When Always On is enabled on a site, Windows Azure will automatically ping your Web Site regularly to ensure that the Web Site is always active and in a warm/running state.  This is useful to ensure that a site is always responsive (and that the app domain or worker process has not paged out due to lack of external HTTP requests). 

It also useful as a way to keep a Web Site active for scenarios where you want to run background code within it irrespective of whether it is actively processing external HTTP customer requests.  We have another new feature we are enabling this week called “Web Jobs” that makes it really easy to now write this background code and run it within a Web Site. I’ll blog more about this feature and how to use it in the next few days.

You can enable Always On support for Web Sites running in Standard mode by navigating to the CONFIGURE tab within the portal, and then toggling the Always On button that is now within it:

image

Monitoring Improvements: Web Sites + SQL Database Alerts

With almost every release we make improvements to our monitoring functionality of Azure services. Today’s update brings two nice new improvements:

  1. Metrics updated every minute for Windows Azure Web Sites
  2. Alerting for more metrics from Windows Azure Websites and Windows Azure SQL Databases

Monitoring Data Every Minute

With today’s release we are now updating statistics on the monitoring dashboard of a Web Site every minute, so you can get much more fresh information on exactly how your website is being used (prior to today the granularity was not as fine grained):

image

Viewing data at this higher granularity can make it easier to observe changes to your website as they happen. No additional configuration is required to get data every minute – it is now automatically enabled for all Azure Websites.

Expanding Alerting

When you create alerts you can now choose between six different services:

  • Cloud Service
  • Mobile Service
  • SQL Database (New Today!)
  • Storage
  • Virtual Machine
  • Web Site (More Metrics Today!)

To get started with Alerting, click on the Management Services extension on the left navigation tab of the the Windows Azure Management Portal:

image

Then, click the Add Rule button in the command bar at the bottom of the screen. This will open a wizard for creating an alert rule. You can see all of the services that now support alerts:

image

New Web Site Alert Metrics

With today’s release we are adding the ability to alert on any metric that you see for a Web Site in the portal (previously we only supported alerts on Uptime and Response Time metrics). Today’s new metrics include support for setting threshold alerts for errors as well as CPU time and total requests:

image

The CPU time and Data Out metric alerts are particularly useful for Free or Shared websites – you can now use these alerts to email you if you’re getting close to exceeding your quotas for a free or shared website (and need to scale up instances).

New SQL Alert Metrics

With today’s release you can also now define alerts for your SQL Databases. For Web and Business tier databases you can setup alert metrics for the Storage for the database.  There are also now additional metrics and alerts for SQL Database Premium (which is currently in preview) such as CPU Cores and IOPS.

Once you’ve set up these new alerts, they behave just like alerts for other services. You’ll be informed when they cross the thresholds you establish, and you can see the recent alert history in the dashboard:

image

Windows Azure Hyper-V Recovery Manager: General Availability Release

I’m excited to announce the General Availability of Windows Azure Hyper-V Recovery Manager (HRM). This release is now live in production, backed by an enterprise SLA, supported by Microsoft Support, and is ready to use for production scenarios.

Windows Azure Hyper-V Recovery Manager helps protect your on premise applications and services by orchestrating the protection and recovery of Virtual Machines running in a System Center Virtual Machine Manager 2012 R2 and System Center Virtual Machine Manager 2012 SP1 private cloud to a secondary location. With simplified configuration, automated protection, continuous health monitoring and orchestrated recovery, Hyper-V Recovery Manager service can help you implement Disaster Recovery and recover applications accurately, consistently, and with minimal downtime.

image

The service leverages Hyper-V Replica technology available in Windows Server 2012 and Windows Server 2012 R2 to orchestrate the protection and recovery of Hyper-V Virtual Machines from one on-premise site to another on-premise site. Application data always travels on your on premise replication channel. Only metadata that is needed (such as names of logical clouds, virtual machines, networks etc.) for orchestration is sent to Azure. All traffic sent to/from Azure is encrypted.

Getting Started

To get started, use the Windows Azure Management Portal to create a Hyper-V Recovery Manager Vault. Browse to Data Services > Recovery Services and click New to create a New Hyper-V Recovery Manager Vault. You can name the vault and specify a region where you would like the vault to be created.

clip_image002

Once the Hyper-V recovery Manager vault is created, you’ll be presented with a simple tutorial that will help guide you on how to register your SCVMM Servers and configure protection and recovery of Virtual Machines.

clip_image004

To learn more about setting up Hyper-V Recovery Manager in your deployment follow our detailed step-by-step guide.

Key Benefits of Hyper-V Recovery Manager

Hyper-V Recovery Manager offers the following key benefits that differentiate it from other disaster recovery solutions:

  • Simple Setup and Configuration: HRM dramatically simplifies configuration and management operations across large number of Hyper-V hosts, Virtual Machines and data-centers.
  • Automated Protection: HRM leverages the capabilities of Windows Server and System Center to provide on-going replication of VMs and ensures protection throughout the lifecycle of a VM.
  • Remote Monitoring: HRM leverages the power and reach of Azure to provide a remote monitoring and DR management service that can be accessed from anywhere.
  • Orchestrated Recovery: Recovery Plans enables automated DR orchestration by sequencing failover of different application tiers and customization with scripts and manual actions.

New Improvements

The Hyper-V Recovery Manager service has been enhanced since the initial October Preview with several nice improvements:

  • Improved Failback Support: The Failback support has been improved in scenarios where the primary host cluster has been rebuilt after an outage.
  • Support for Kerberos based Authentication: Cloud configuration now allows selecting Kerberos based authentication for Hyper-V Replica. This is useful in scenarios where customers want to use 3rd party WAN optimization and compression and have AD trust available between primary and secondary sites.
  • Support for Upgrade from VMM 2012 SP1 to VMM 2012 R2: HRM service now supports upgrades from VMM 2012 SP1 to VMM 2012 R2.
  • Improved Scale: The UI and service has been enhanced for better scale support.

Please visit Windows Azure web site for more information on Hyper-V Recovery Manager. You can also refer to additional product documentation. You can visit the HRM forum on MSDN for additional information and engage with other customers.

Mobile Services: Support for SenchaTouch

I’m excited to announce that in partnership with our friends at Sencha, we are today adding support for SenchaTouch to Windows Azure Mobile Services. SenchaTouch is a well know HTML/JavaScript-based development framework for building cross-platform mobile apps and web sites. With today’s addition, you can easily use Mobile Services with your SenchaTouch app.

You can download Windows Azure extension for Sencha here, configure Sencha loader with the location of the azure extension, and add Azure package to your app.json file:

{ name : "Basic", requires : [ "touch-azure"]}

Once you have the Azure extension added to your Sencha project, you can connect your Sencha app to your Mobile Service simply by adding the following initialization code:

Ext.application({

    name: 'Basic',

    requires: ['Ext.azure.Azure'],

    azure: {

        appKey: 'myazureservice-access-key',

        appUrl: 'myazure-service.azure-mobile.net'

    },

    launch: function () {

        // Call Azure initialization

        Ext.Azure.init(this.config.azure);

    }

});

From here on you can data bind your data model to Azure Mobile Services, authenticate users and use push notifications. Follow this detailed Getting Started tutorial to get started with SenchaTouch and Mobile Services. Read more detailed documentation at Mobile Services Sencha extension resources page.

Windows Azure Now Validated for PCI DSS Compliance

We are very excited to announce that Windows Azure has been validated for compliance with the Payment Card Industry (PCI) Data Security Standards (DSS) by an independent Qualified Security Assessor (QSA).

The PCI DSS is the global standard that any organization of any size must adhere to in order to accept payment cards, and to store, process, and/or transmit cardholder data. By providing PCI DSS validated infrastructure and platform services, Windows Azure delivers a compliant platform for you to run your own secure and compliant applications. You can now achieve PCI DSS certification for those applications using Windows Azure.

To assist customers in achieving PCI DSS certification, Microsoft is making the Windows Azure PCI Attestation of Compliance and Windows Azure Customer PCI Guide available for immediate download.

Visit the Trust Center for a full list of in scope features or for more information on Windows Azure security and compliance.

Summary

Today’s release includes a bunch of great features that enable you to build even better cloud solutions.  If you don’t already have a Windows Azure account, you can sign-up for a free trial and start using all of the above features today.  Then visit the Windows Azure Documentation Center to learn more about how to build apps with it.

Hope this helps,

Scott

P.S. In addition to blogging, I am also now using Twitter for quick updates and to share links. Follow me at: twitter.com/scottgu

Categories: Architecture, Programming

Vedis - An Embedded Implementation of Redis Supporting Terabyte Sized Databases

I don't know about you, but when I first learned about Redis my initial thought was wow, why hasn't anyone done this before? My next thought was why put this functionality in a separate process? Why not just embed it in your own server code and skip the network path completely? Especially in a Service Oriented Architecture there's no need for an extra hop or extra software installation and configuration.

Now you can embed Redis-like code directly into your server with Vedis - an embeddable datastore C library built with over 70 commands similar in concept to Redis but without the networking layer since Vedis run in the same process of the host application. It's transactional, cross platform, thread safe, key-value, supports terabyte sized databases, has a GPL-like license (which isn't great for commercial apps), and supports an on-disk as well as in-memory datastore.

More about Vedis:

Categories: Architecture

Announcing The Texas Chapter of the Association of Enterprise Architects

Mike Walker's Blog - Wed, 01/15/2014 - 06:31

 Texas Association of Enterprise Architects

I would like to be the first to announce the Texas Chapter of the Association of Enterprise Architects!

Myself and the rest of the the leadership have been working very hard to create this chapter. This post provides a quick overview of the Global Association of Enterprise Architects and the Texas Chapter of the AEA along with an invite to the first meeting in Austin Texas.

 

Introduction to the Global Association of Enterprise Architects Association of Enterrpise Archtiects logo

  This chapter is part of a greater global organization called the Association of Enterprise Architects. The Association of Enterprise Architects (AEA) is the only definitive professional organization for Enterprise Architects with over 35,000 professionals world-wide.

Its goals are:

  • Raise the status of the EA profession
  • Foster an already vibrant EA community
  • Access to EA thought leadership
  • Increase market value of professionals

 

Introduction of the Texas AEA

These were all great goals and I wanted to bring this level of professionalism to Texas. In this state there is a vibrant community of Texas professionals, vendors and consortiums offers thought leadership, visions for the future, and proven practices based on reality of today. Building on the Global AEA Texas has the following objectives:

 Texas Association of Enterprise Architects

 

This chapter will strive to have the following characteristics to ensure a vibrant and highly interactive community forum. We will be…

  • Regular. With quarterly and monthly meetings starting first in Austin, TX and then branching out to Houston, Dallas and San Antonio.
  • Impactful. Through full day events we hope to draw in top speakers in the architecture community.
  • Highly Interactive . These meetings and events focus first on the community and take the best from a conferences and a user groups.
  • High Quality. The key to a vibrant and sustainable community is to make sure there is high quality discussions and interactions.

We have been lucky with some top notch sponsorship support and I would like to personally thank all these vendors for helping to make this happen! With out your support it would be difficult for us to get this moving.

 

 Texas Association of Enterprise Architects

 

 

Join Us!

So what’s left to say, but come on and join us in this professional network. Myself and the rest of the Texas AEA members would be honored to have you join.

You can join by coming to our first meeting in Austin Texas on January 30th. You can find the invite here.

 

If your  still not sold, below are a few reasons why I think it’s important for all of you to consider:

  • The Network. There is already 700 members of the AEA w/out an established chapter.
  • Access. You have the ability to access not only the Texas chapter but others along with job boards and other collaborative features.
  • Amplification. Amplify YOUR voice in the community. Show your and your company’s leadership in the market.
  • SIG’s. Special interest groups are a great way to get smart and passionate people together to create exciting and new innovative ideas.
  • And much more…
Categories: Architecture

SharePoint VPS solution

Microsoft SharePoint is an ideal solution for companies who have multiple offices and staff members who are on the move. Using SharePoint, documents and other materials can be easily shared with both colleagues and managers. Other features include advanced document management, which allows users to virtually check out a document, modify it or just read it, then check in the document again. This allows managers/company owners to see exactly when their staff members are working and just what they are doing. When combined with a highly customizable workflow management system and group calendars, SharePoint can improve the way in which your company functions and operates.

However, many organizations are observed to be failing with SharePoint implementation. So with this article, we are trying to make it simpler for organizations in-house IT administrators to help implement SharePoint over a virtual server environment.

Here we are going to see following key points:

Categories: Architecture