Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
Software Development Blogs: Programming, Software Testing, Agile Project Management
Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!
TL;DR: There’s a lot of unused space in North America. Yet cities like San Francisco are becoming ever more expensive because of a bubble created by high tech jobs that seemingly can be done anywhere. Historically cities are built around resources that provide some service to humans. The age of infrastructure rising around physical resources is declining while the age of digital resource exploitation is rising. Cities are still valuable because they are amazing idea and problem solving machines. How about we create thousands of new Maker Cities in the vast emptiness that is North America and build them around digital resources like bandwidth, compute power, Atomically-Precise Manufacturing (AMP), and all things future and bright?Observation Number One: There’s lots of empty space out there.
There's been some interesting discussion over the past fews days about Leanpub, both on Twitter and blogs. Jurgen Appelo posted Why I Don't Use Leanpub and Peter Armstrong responded. I think the biggest selling points of Leanpub as a publishing platform from an author's perspective may have been lost in the discussion. So, here's why my take on why I use Leanpub for Software Architecture for Developers.Some history
I pitched my book idea to a number of traditional publishing companies in 2008 and none of them were very interested. "Nice idea, but it won't sell" was the basic summary. A few years later I decided to self-publish my book instead and I was about to head down the route of creating PDF and EPUB versions using a combination of Pages and iBooks Author on the Mac. Why? Because I love books like Garr Reynolds' Presentation Zen and I wanted to do something similar. At first I considered simply giving the book away for free on my website but, after Googling around for self-publishing options, I stumbled across Leanpub. Despite the Leanpub bookstore being fairly sparse at the start of 2012, the platform piqued my interest and the rest is history.The headline: book creation, publishing, sales and distribution as a service
I use Leanpub because it allows me to focus on writing content. Period. The platform takes care of creating and selling e-books in a number of different formats. I can write some Markdown, sync the files via Dropbox and publish a new version of my book within minutes.Typesetting and layout
I frequently get asked for advice about whether Leanpub is a good platform for somebody to write a book. The number one question to ask is whether you have specific typesetting/layout needs. If you want to produce a "Presentation Zen" style book or if having control of your layout is important to you, then Leanpub isn't for you. If, however, you want to write a traditional book that mostly consists of words, then Leanpub is definitely worth taking a look at.
Leanpub uses a slightly customised version of Markdown, which is a super-simple language for writing content. Here's an example of a Markdown file from my book, and you can see the result in the online sample of my book. Leanpub does allow you to tweak things like PDF page size, font size, page breaking, section numbering, etc but you're not going to get pixel perfect typesetting. I think that Leanpub actually does a pretty fantastic job of creating good looking PDF, EPUB and MOBI format ebooks based upon the very minimal Markdown. This is especially true when you consider the huge range of ebook reader software across PCs, Macs, Android devices, Apple devices, Kindles, etc. Plus the readers themselves can mess with the fonts/font sizes too.
It's like building my own server at Rackspace versus using a "Platform as a Service" such as Cloud Foundry. You need to make a decision about the trade-off between control and simplicity/convenience. Since authoring isn't my full-time job and I have lots of other stuff to be getting on with, I'm more than happy to supply the content and let Leanpub take care of everything else for me.Toolchain
My toolchain as a Leanpub author is incredibly simple: Dropbox and Mou. From a structural perspective, I have one Markdown file per essay and that's basically it. Leanpub does now provide support for using GitHub to store your content and I can see the potential for a simple Leanpub-aware authoring tool, but it's not rocket science. And to prove the point, a number of non-technical people here in Jersey have books on Leanpub too (e.g. Thrive with The Hive and a number of books by Richard Rolfe).Iterative and incremental delivery
Before starting, I'd already decided that I'd like to write the book as a collection of short essays and this was cemented by the fact that Leanpub allows me to publish an in-progress ebook. I took an iterative and incremental approach to publishing the book. Rather than starting with essay number one and progressing in order, I tried to initially create a minimum viable book that covered the basics. I then fleshed out the content with additional essays once this skeleton was in place, revisiting and iterating upon earlier essays as necessary. I signed up for Leanpub in January 2012 and clicked the "Publish" button four weeks later. That first version of my book was only about ten pages in length but I started selling copies immediately.Variable pricing and coupons
Another thing that I love about Leanpub is that it gives you full control over how you price your book. The whole pricing thing is a balancing act between readership and royalties, but I like that I'm in control of this. My book started out at $4.99 and, as content was added, that price increased. The book now currently has a minimum price of $20 and a recommended price of $30. I can even create coupons for reduced price or free copies too. There's some human psychology that I don't understand here, but not everybody pays the minimum price. Far from it, and I've had a good number of people pay more than the recommend price too. Leanpub provides all of the raw data, so you can analyse it as needed.An incubator for books
As I've already mentioned, I pitched my book idea to a bunch of regular publishing companies and they weren't interested. Fast-forward a few years and my book is the currently the "bestselling" book on Leanpub this week, fifth by lifetime earnings and twelfth in terms of number of copies sold. I've used quotes around "bestselling" because Jurgen did. ;-)
In his blog post, Peter Armstrong emphasises that Leanpub is a platform for publishing in-progress ebooks, especially because you can publish using an iterative and incremental approach. For this reason, I think that Leanpub is a fantastic way for authors to prove an idea and get some concrete feedback in terms of sales. Put simply, Leanpub is a fantastic incubator for books. I know of a number of books that were started on Leanpub have been taken on by traditional publishing companies. I've had a number of offers too, including some for commercial translations. Sure, there are other ways to publish in-progress ebooks, but Leanpub makes this super-easy and the barrier to entry is incredibly low.The future for my book?
What does the future hold for my book then? I'm not sure that electronic products are ever really "finished" and, although I consider my book to be "version 1", I do have some additional content that is being lined up. And when I do this, thanks to the Leanpub platform, all of my existing readers will get the updates for free.
I've so far turned down the offers that I've had from publishing companies, primarily because they can't compete in terms of royalties and I'm unconvinced that they will be able to significantly boost readership numbers. Leanpub is happy for authors to sell their books through other channels (e.g. Amazon) but, again, I'm unconvinced that simply putting the book onto Amazon will yield an increased readership. I do know of books on the Kindle store that haven't sold a single copy, so I take "Amazon is bigger and therefore better" arguments with a pinch of salt.
What I do know is that I'm extremely happy with the return on my investment. I'm not going to tell you how much I've earned, but a naive calculation of $17.50 (my royalty on a $20 sale) x 4,600 (the total number of readers) is a little high but gets you into the right ballpark. In summary, Leanpub allows me focus on content, takes care of pretty much everything and gives me an amazing author royalty as a result. This is why I use Leanpub.
What if your work could be your ultimate platform? âŠ your ultimate channel for your growth and greatness?
We spend a lot of time at work.
For some people, work is their ultimate form of self-expression.
For others, work is a curse.
Nobody stops you from using work as a chance to challenge yourself, to grow your skills, and become all that youâre capable of.
But thatâs a very different mindset than work is a place you have to go to, or stuff you have to do.
When you change your mind, you change your approach. And when you change your approach, you change your results. But rather than just try to change your mind, the ideal scenario is to expand your mind, and become more resourceful.
You can do so with quotes.Grow Your âWork Intelligenceâ with Inspirational Work Quotes
In fact, you can actually build your âwork intelligence.â
Here are a few ways to think about âintelligenceâ:
In Rich Dad, Poor Dad, Robert Kiyosaki, says, âintelligence is the ability to make finer distinctions.â And, Tony Robbins, says âintelligence is the measure of the number and the quality of the distinctions you have in a given situation.â
If you want to grow your âwork intelligenceâ, one of the best ways is to familiarize yourself with the best inspirational quotes about work.
By drawing from wisdom of the ages and modern sages, you can operate at a higher level and turn work from a chore, into a platform of lifelong learning, and a dojo for personal growth, and a chance to master your craft.
You can use inspirational quotes about work to fill your head with ideas, distinctions, and key concepts that help you unleash what youâre capable of.
To give you a giant head start and to help you build a personal library of profound knowledge, here are two work quotes collections you can draw from:
Letâs practice. This will only take a minute, and if you happen to hear the right words, which are the keys for you, your insight or âah-haâ can be just the breakthrough that you needed to get more of your work, and, as a result, more out of life (or at least your moments.)
Here is a sample of distinct ideas and depth that you use to change how you perceive your work, and/or how you do your work:
For more ideas, take a stroll through my inspirational work quotes.
As you can see, there are lots of ways to think about work and what it means. At the end of the day, what matters is how you think about it, and what you make of it. Itâs either an investment, or itâs an incredible waste of time. You can make it mundane, or you can make it matter.The Pleasant Life, The Good Life, and The Meaningful Life
Hereâs another surprise about work. You can use work to live the good life. According to Martin Seligman, a master in the art and science of positive psychology, there are three paths to happiness:
In The Pleasant Life, you simply try to have as much pleasure as possible. In The Good Life, you spend more time in your values. In The Meaningful Life, you use your strengths in the service of something that is bigger than you are.
There are so many ways you can live your values at work and connect your work with what makes you come alive.
There are so many ways to turn what you do into service for others and become a part of something thatâs bigger than you.
If you havenât figured out how yet, then dig deeper, find a mentor, and figure it out.
You spend way too much time at work to let your influence and impact fade to black.You Might Also Like
I presented big data to Amdocs’ product group last week. One of the sessions I did was recorded so I might be able to add here later. Meanwhile you can check out the slides.
Note that trying to keep the slide visual I put some of the information is in the slide notes and not on the slides themselves.
Google’s Jeffrey Dean and Sanjay GhemawatÂ filed theÂ patent requestÂ and published theÂ map/reduce paper Â 10 year ago (2004). According to WikiPedia Doug Cutting and Mike Cafarella created Hadoop, with its own implementation of Map/Reduce, Â one year later at Yahoo – both these implementations were done for the same purpose – batch indexing of the web.
Back than, the web beganÂ its “web 2.0″ transition, pages became more dynamic , people began to create more content – so an efficient way to reprocess and build the web index was needed and map/reduce was it. Web Indexing was a great fit for map/reduce since the initial processing of each source (web page) is completely independent from any other – i.e. Â a very convenient map phase and you need Â to combine the results to build the reverse index. That said, even the coreÂ google algorithm – Â the famous pagerankÂ is iterative (so less appropriate for map/reduce), not to mention that Â as the internet got bigger and theÂ updates became more and more frequent map/reduceÂ wasn’t enough. Again Google (who seem to beÂ consistentlyÂ few years ahead of the industry) began coming up with alternatives like Google PercolatorÂ orÂ Â Google DremelÂ (both papers were published in 2010, Percolator was introduced at that year, andÂ dremel has been used in Google since 2006).
So now, it is 2014, and it is time for the rest of us to catch up with Google and get over Map/Reduce and Â for multiple reasons:
In my opinion, Map/Reduce is an idea whose time has come and gone – it won’t die in a day or a year, there is still a lot of working systems that use it and the alternatives are still maturing. I do think, however, that if you need to write or implement something new that would build on map/reduce – you should use other option or at the very least carefully consider them.
So how is this change going to happenÂ ? Â Luckily, Hadoop has recently adopted YARN (you can see my presentation on it here), which opens up the possibilities to go beyond map/reduce without changing everything … even though in effect, Â a lotÂ will change. Note that some of the new options do have migration paths and also we still retain theÂ Â access to all that “big data” we have in Hadoopm as well as the extended reuse of some of the ecosystem.
The first type of effort to replace map/reduce is to actually subsume it by offering moreÂ Â flexible batch. After all saying Map/reduce is not relevant, deosn’t mean that batch processing is not relevant. It does mean that there’s a need to more complex processes. There are two main candidates here Â TezÂ and SparkÂ where Tez offers a nice migration path as it is replacing map/reduce as the execution engine for both Pig and Hive and Spark has a compelling offer by combining Batch and Stream processing (more on this later) in a single engine.
The second type of effort or processing capability that will help kill map/reduce is MPP databases on Hadoop. Like the “flexible batch” approach mentioned above, this is replacing a functionality that map/reduce was used for – unleashing the data already processed and stored in Hadoop. Â The idea here is twofold
Efforts in this arena include Impala from Cloudera, Hawq from Pivotal (which is essentially greenplum over HDFS), startups like Hadapt or even Actian trying to leverage their ParAccel acquisition with the recently announced Actian Vector . Hive is somewhere in the middle relying on Tez on one hand and using vectorization and columnar format (Orc) Â on the other
The ThirdÂ type of processing that will help dethrone Map/Reduce is Stream processing. Unlike the two previous types of effort this is covering a ground the map/reduce can’t cover, even inefficiently. Stream processing is about Â handling continuous flow of new data (e.g. events) and processing them Â (enriching, aggregating, etc.)Â Â them in seconds or less. Â The two major contenders in the Hadoop arena seem to be Spark Streaming and StormÂ though, of course, there are several other commercial and open source platforms that handle this type of processing as well.
In summary – Map/Reduce is great. It has served us (as an industry) for a decade but it is now time to move on and bring the richer processing capabilities we have elsewhere to solve our big data problems as well.
Last note Â – I focused on Hadoop in this post even thought there are several other platforms and tools around. I think that regardless if Hadoop is the best platform it is the one becoming the de-facto standard for big data (remember betamax vs VHS?)
One really, really last note – if you read upÂ to here, and you are a developer living in Israel, and you happen to be looking for a job – Â I am looking for another developer to join my Technology Research team @ Amdocs. If you’re interested drop me a note: arnon.rotemgaloz at amdocs dot com or via my twitter/linkedin profiles
*esp. in regard to analytical queries – operational SQL on hadoop with efforts like Â Phoenix ,IBM’s BigSQL or Splice Machine are also happening but that’s another story
illustration idea found inÂ Â James Mickens’s talk in Monitorama 2014 – Â (which is, by the way, a really funny presentation – go watch it) -ohh yeah… and pulp fiction :)
I did a short overview of Hadoop YARN to our big data development team. The presentation covers the motivation for YARN, how it works and its major weaknesses
You can watch/download on slideshare
My review copy has arrived and I’ll be reviewing it just as soon as I can, but in the meantime if you’d like more information about this new book go toÂ http://bit.ly/wpNF1J