Spark Forum - Data Science

Spark basic interview questions in a mock interview.

1) Why you shifted from map reduce development to spark development?
2) How Spark Engine is different from Hadoop Map Reduce engine?
3) What are the steps for spark jobs optimization?
4) What is executor and executor core? Reference in terms of process & threads
5) How to you identify that your hive script is slow?
6) When do we use partitioning and bucketing in hive?
7) Small file problem in hive ? ---> Skewiness
😎 How do you improve high cardinality issue in dataset? In resect of Hive.
9) How do you care code merging with other teams, explain your development process?
10) Again, Small files issue in Hadoop ?
11) Metadatasize of hadoop ?
12) How spark is differentiated from Map Reduce?
13) In a class having 3 fields name,age,salary & you are creating series of objects from this class? How do you compare the object ----(I didn't got the question exactly)
14) Scala : what is === in joins conditions? What does it means?