Data Engineer

Responsibilities:
•    Design, develop, and maintain software solutions using Python.
•    Developed Python and pyspark programs for data analysis.. Good working experience with python to develop Custom Framework for generating of rules (just like rules engine).
•    Developed Python code to gather the data from HBase and designs the solution to implement using Pyspark. Apache Spark Data Frames/RDD's were used to apply business transformations and utilized Hive Context objects to perform read/write operations..
•    Develop back-end services, APIs, and integrate databases.
•    Optimize applications for performance, security, and maintainability.
•    Troubleshoot and resolve software defects and issues.
Required Skills:
•    Strong proficiency in Python and Spark