Big Data for Freshers

[Ashish_Ahuja]

Hi, I am an engineering student and wanna know how to start working in the field of Big Data ? Cause I wanna learn more about it before stepping into the industry. I have a great interest in Big Data technologies.

[SergeyKostrov]

Here is my advise... Start programming! For example, in C or C++ write a simple program: Create an array of 1 Giga Elements of random Single-Precision values ( 4 Giga Bytes of memory in total ) and sort it with some sorting algorithm, like Merge, Heap or Quick sort.

[yaswanth_k_]

Relational database management systems and desktop statistics and visualization packages often have difficulty handling big data. The work instead requires "massively parallel software running on tens, hundreds, or even thousands of servers". What is considered "big data" varies depending on the capabilities of the users and their tools, and expanding capabilities make Big Data a moving target. Thus, what is considered "big" one year becomes ordinary later. "For some organizations, facing hundreds of gigabytes of data for the first time may trigger a need to reconsider data management options. For others, it may take tens or hundreds of terabytes before data size becomes a significant consideration

[yaswanth_k_ · Accepted solution]

some of the skills you may required for "BIG DATA"

1. Apache Hadoop

Sure, it’s entering its second decade now, but there’s no denying that Hadoop had a monstrous year in 2014 and is positioned for an even bigger 2015 as test clusters are moved into production and software vendors increasingly target the distributed storage and processing architecture. While the big data platform is powerful, Hadoop can be a fussy beast and requires care and feeding by proficient technicians. Those who know there way around the core components of the Hadoop stack–such as HDFS, MapReduce, Flume, Oozie, Hive, Pig, HBase, and YARN–will be in high demand.

2. Apache Spark

If Hadoop is a known quantity in the big data world, then Spark is a black horse candidate that has the raw potential to eclipse its elephantine cousin. The rapid rise of the in-memory stack is being proffered as a faster and simpler alternative to MapReduce-style analytics, either within a Hadoop framework or outside it. Best positioned as one of the components in a big data pipeline, Spark still requires technical expertise to program and run, thereby providing job opportunities for those in the know.

3. NoSQL

On the operational side of the big data house, distributed, scale-out NoSQL databases like MongoDB and Couchbase are taking over jobs previously handled by monolithic SQL databases like Oracle and IBM DB2. On the Web and with mobile apps, NoSQL databases are often the source of data crunched in Hadoop, as well as the destination for application changes put in place after insight is gleaned from Hadoop. In the world of big data, Hadoop and NoSQL occupy opposite sides of a virtuous cycle.

4. Machine Learning and Data Mining

People have been mining for data as long as they’ve been collecting it. But in today’s big data world, data mining has reached a whole new level. One of the hottest fields in big data last year is machine learning, which is poised for a breakout year in 2015. Big data pros who can harness machine learning technology to build and train predictive analytic apps such as classification, recommendation, and personalization systems are in super high demand, and can command top dollar in the job market.

5. Statistical and Quantitative Analysis

This is what big data is all about. If you have a background in quantitative reasoning and a degree in a field like mathematics or statistics, you’re already halfway there. Add in expertise with a statistical tool like R, SAS, Matlab, SPSS, or Stata, and you’ve got this category locked down. In the past, most quants went to work on Wall Street, but thanks to the big data boom, companies in all sorts of industries across the country are in need of geeks with quantitative backgrounds.

6. SQL

The data-centric language is more than 40 years old, but the old grandpa still has a lot of life yet in today’s big data age. While it won’t be used with all big data challenges (see: NoSQL above), the simplify of Structured Query Language makes it a no-brainer for many of them. And thanks to initiatives like Cloudera‘s Impala, SQL is seeing new life as the lingua franca for the next-generation of Hadoop-scale data warehouses.

7. Data Visualization

Big data can be tough to comprehend, but in some circumstances there’s no replacement for actually getting your eyeballs onto data. You can do multivariate or logistic regression analysis on your data until the cows come home, but sometimes exploring just a sample of your data in a tool like Tableau or Qlikview can tell you the shape of your data, and even reveal hidden details that change how you proceed. And if you want to be a data artist when you grow up, being well-versed in one or more visualization tools is practically a requirement.

8. General Purpose Programming Languages

Having experience programming applications in general-purpose languages like Java, C, Python, or Scala could give you the edge over other candidates whose skill sets are confined to analytics. According to Wanted Analytics, there was a 337 percent increase in the number of job postings for “computer programmers” that required background in data analytics. Those who are comfortable at the intersection of traditional app dev and emerging analytics will be able to write their own tickets and move freely between end-user companies and big data startups.

9. Creativity and Problem Solving

No matter how many advanced analytic tools and techniques you have on your belt, nothing can replace the ability to think your way through a situation. The implements of big data will inevitably evolve and new technologies will replace the ones listed here. But if you’re equipped with a natural desire to know and a bulldog-like determination to find solutions, then you’ll always have a job offer waiting somewhere.

[Ashish_Ahuja]

Thanks Mr. Sergey Kostrov and Mr.yaswanth k. for taking time out and helping me !! I am gonna start right away with everything you have suggested. Kudos !!

[Ashish_Ahuja]

Sure yaswanth k. sir !! Thank You !!

[Ashish_Ahuja]

Good to hear that John R. and thanks a ton for your suggestion.

[Amir_K_2]

Thanks for the explained view ,Ashisha :)

[Aulia_R_]

thanks for the post were very helpful. I am a student who is studying also about big data and this forum is very helpful.

[Ashish_Ahuja]

Our pleasure Aulia R. :)

[Amar_W_]

thank you yaswanth k for your post about some of the skills you may required for "BIG DATA" .it was very helpfull.

[Thomson_P]

I'm trying to learn Python for machine learning. As I go deeper and deeper I come across more tools and SDK's I have no idea what to do. I'm confused with stuffs like Theano, NumPhy etc.I hope that I'll understand all these sometime

[Walker__Terry]

How is Big data fairing among students these days? Is there a lot of interest still?

[gowsi__saran]

This is an awesome post.Really very informative and creative contents. These concept is a good way to enhance the knowledge.I like it and help me to development very well.Thank you for this brief explanation and very nice information.Well, got a good knowledge.

[IPmedia1]

Very informative discussion. Thanks for sharing guys.

[Murugesh_Manthiramoo]

Very useful topic I think. I hope beginning with python is an excellent option

[James__Patrick]

This has been quite informative, not just the post itself but also on the intelligent comments here. Thanks for sharing guys.

Master KG

[infotech__rk]

Nice idea. It is better for you to start doing practice in the field of coding technology. You can learn such type of any IT field subject by seeing code of custom-developed software. Such as a code of custom-developed ERP So if you start learning today by your self you can definitely learn any concept Big data easily. You can learn it with the help of youtube learning videos.