Earth be still. Big data has lost its luster. Could it be that analyzing terabytes, exabytes and zettabytes of information won’t make us smarter … or, even worse, could it make us wrong? We’re beginning to see headlines like “Google and the flu: how big data will help us make gigantic mistakes” in the Guardian, “Eight (No, Nine!) Problems With Big Data” in the New York Times and “Big Data: Are we making a big mistake?” on author and Financial Times columnist’s Tim Harford’s site.
I found out today about toptal.com thanks to the job offer which arrived on my LinkedIn mailbox from Tania L. It’s a website which reminded me in the beginning of a oDesk alternative, but it makes for more direct connection on the line between engineers and employers. The are no job offer listing here, toptal makes sure that a company will find proper engineers for its ongoing tasks.
At Toptal, you won’t have to throw product requirements over a wall, deal with multiple middlemen managing your team, or pay a flat project fee. Instead, you will be connected with a superb remote engineer, who will interface with your team as an internal engineer working from home would.
It’s another example for me when a company focused on analyzing and dealing with job market searches for a Data Scientist. Recently I saw Data science internships in the Monster co. (yup, the job search portal). So despite the recession job seek professions seems fine and they have a lot of multidimensional data to analyze.
Well, first of all, the big data used by researches must be given from hand to hand (this ad-hoc is the worst solution 🙂 ) to enable cooperation, teamwork etc.
Secondly, research financial grants often requires that data and code must be open-source and available freely.
Cloud computing of course is not always possible, and that’s the reason of creating this post. Researches use vast majority of statistical and data-mining tools which operate on local resources or at most remote databases (i.e. RapidMiner, R).
Dropbox and public folder – especially good solution if you have gained a lot of extra space because of limited promotions (i.e. after buying a Samsung smartphone). I had around 100GB in a peak moment and was called a Dropbox Guru twice. After putting data into “Public” folder, you have there a thing in right-click-menu called” copy public link” which allows to download data anywhere from the world.
Amazon AWS cloud – popular within companies, but solution payable depending on the transfer used