The recent spat between Peoplebrowsr and Twitter raised the profile of one area of the social tech scene that is sometimes ignored – social scoring. Peoplebrowsr own Kred a social analytics company which offers social analytics and scoring to individuals and companies. Klout is another famous example of this and there are also Adobe Social Analytics, TrackSocial, SproutSocial, Kontagent and many others in this space. You might even remember BackType, who were famously acquired by Twitter in July 2011.
The unholy combination of social graphs, big data, nosql and massively parallel algorithms sounds like a winning hand in recruitment bullshit bingo but these paradigms/technologies form the core of this industry where the interactions and relationships between interconnected entities (people and companies to you and me) are parsed and analysed.
If you are a brand, then these companies offer real insight into your presence in the social media. If you are a person, then you probably don’t give a monkeys. Yes yes I know that having a Klout score of over 50 gets you into the Singapore Air lounge at SFO but lets be honest, if you are the sort of person that can take advantage of that benefit you are highly likely to have access to lounges anyway so hardly a deal clincher.
I was in the Facebook offices in London recently. Very cool offices in a very nice location but what one of the things that struck me most was the signage on the walls.
I have to say that I absolutely love this.
It’s one thing to hear this at conferences (which sometimes seem to exist for the express purpose of making us aware of our own incompetence!) or from coworkers who have your back anyway. It’s another thing to have this as fundamental aspect of the culture corporate and just a million miles away from the lip service that so many companies pay to this idea. You know the sort of company, they maybe have a nice motivational picture of an eagle along with a pithy comment about attitude and altitude and think they are being daring. If you are lucky your direct boss embraces the idea that being able to fail is a good thing and if you are really lucky your boss’s boss does too but the whole company? Really?
Big Data, the processing of data sets that do not fit on a single computer, has come of age. It’s not just the level of interest shown at conferences like Strata but also the types of people participating. Sure there are loads of companies out there with products in this space but there are also plenty of end users coming forward and many of these are outside of technology companies. At the London version, one of the speakers was Ben Goldacre, doctor and author of the awesome Bad Science, who discussed the impact of missing data which is a huge issue for medical studies. Even the Whitehouse has weighed in on behalf of Big Data and emphasized its importance to business.
If you are going to use a technology I’m a big fan of going to the source and thankfully in this space, a lot of the published work on this is freely available. So I’ve collected some of the papers that are key to this area: five are about Big Data itself and the bonus one is about operational monitoring for massively distributed systems.
When I first moved into a position of leading developers I experienced a fair amount of good natured teasing that I’d do my best programming via Outlook and Omnigraffle. While this has been true (thankfully only to a point!) it has been outweighed by the ability to have a far greater say into the resourcing, architecture and implementation of projects and how my company does its work.
One thing that I’ve become painfully aware of is that the operational aspect of this position has a larger ‘people management’ aspect to it than I had previously thought. This is especially true when you extend your view to include recruitment, appraisals, motivating staff and the like and not just setting up your teams to deliver quality code as fast as they can. The soft skills of leadership tend not to come easily to those with a natural technical focus (describing almost every developer ever) so it only makes sense to take advantage of psychology research in the areas of skills assessment and motivation. These is a wealth of knowledge out there and I thought I would discuss a few of the more interesting/useful ones that I have come across.
It’s unarguable that Continuous Delivery has gone from being just a CTO friendly buzzword to a central requirement for a high performance delivery team. It’s no longer cutting edge to merely check in your code to source control and have Jenkins or other continuous integration box run the unit tests. You have to be able to get that code out into a live environment as fast as possible and that means Continuous Delivery. The ability to deliver code into production at will has a direct effect on your bottom line but to do this effectively you need two things, 1) understand the important areas of functionality that the customers really use and 2) be able to test these areas as quickly and easily as possible. The first is a business issue but the second boils down to automating your testing.
The trouble is that most of the industry holds on to a quality assurance process that is directly at odds with this. The reasons are mostly historical but companies have had varying levels of success in the drive to automate QA. The level varies on how highly the company values this ability. So what levels of QA do we commonly see?
The Velocity EU conference is over for this year and what an amazing time. It’s my third Velocity Conference and it just keeps getting better.
There were brilliant talks everywhere. Newer speakers like Andrew Brockhurst (BBC), Dave Nolan and Mark Jennings (Lonely Planet), Mike Krieger (Instagram), Brian Whitman (EchoNest) alongside veterans like Theo Schlossnagle (OmniTI), Artur Bergman (Fastly), Tim Morrow (Betfair), Steve Souders (Google), Mike Rembetsy, Patrick McDonnell and John Allspaw (Etsy.com) and everyone with a compelling message. Even I finally got up on stage and did a talk along with Abe (slides here) where hopefully I didn’t embarrass myself too much. It’s not a joke that hanging out in the hallway really is a fourth track. Along with the Operations and two Performance tracks, with so many smart people the clever kind of rubs off on you as you wander around in search of coffee. You are so much better just for having been there.
This post on the temptation of data raised an interesting aspect of Big Data: that we run the risk of being overwhelmed by data and that organisations and investors are looking too hard for the one piece of data that will be the key to success. Drowning in data is a real risk to an enterprise and something that great leaders are aware of ( lesson 3 in this awesome leadersip slidedeck from Colin Powell – “Experts often posses more data than judgement”) but being swamped by the data is not the only problem.
Another problem with casting the net of big data wide in this way is that eventually you will end up finding some sort of pattern, any sort of pattern if you keep looking hard enough (there’s an XKCD for this). The human brain is a fantastic pattern recognition system and is easily fooled (see Pareidolia) and it’s far too easy to make mistakes like confusing correlation with causation.
Graduation time again. Newly minted grads will be taking to the streets looking for a job and many may well have applied or being looking to apply for graduate programs at companies both large and small.
These programs may look to rotate recent grads through the different departments over an 18 month to 2 year period to allow them to find their ideal niche or just be a direct pipeline into a single department. Given the current economy, even a half dozen spaces on a graduate program are going to have several hundred application for each space.
This raises an obvious question for a technology company looking to recruit into their development organisation: how the hell are HR, who we all know lack the understanding of what makes a good technologist, able to winnow this ridiculous volume down to something where developers can get involved? At my current client we would love to be able to interview every candidate. Given that we like to spend at least half a day doing an initial assessment, that just won’t scale. Because of this I was asked to help come up with a solution that could be completed by the candidates and/or the HR folk to generate a way of ranking candidates. So: non-technical people assessing technical people? It’s going to have to be some sort of list.
“There is never any shame in being wrong, only in being too ignorant to learn why you were wrong.”
NoSQL is a hot topic right now; as long as you don’t need ACID guarantees or complex joins you can have a persistence store that is faster, scales better, allows greater schema flexibility and all at a lower comparable cost than a relational database. The number of companies looking to use NoSQL has grown massively and the number of NoSQL solutions looking to feed this grown have blossomed also.
In the eye of this storm are three sets of individuals On one side we have the developers desperate to own the full stack from web app to data store, in the middle are the Ops guys and DBAs used to owning and running the persistence stores and on the other side are the vendors selling their wares. One group focuses on delivering new features as quickly as possible, another ensures that they run smoothly and can be recovered as and when they go bang and the third are deluging these other two groups with an almost impossible amount of information to make sure that their solution is the one being used.