Conduct POCs (proof of concept) and pilots to try new solutions and provide summary of results.
Participate in detailed technical design, development, implementation and support of Big Data applications.
Maintain standards compliance and ensure development artifacts are in alignment with patterns/ frameworks designed by software engineering teams.
Required Skills
Familiarity with Spark programming paradigms (batch and stream-processing). Understanding of different data abstraction objects used in spark for different use cases, use of optimal data format and other optimization techniques.
Strong programming skills in at least one of the following languages: Java, Scala.
Familiarity with a scripting language like Python as well as Unix/Linux shells.
Strong knowledge of writing optimized Spark and Hive sql and experience to tune poor performing queries.
Outstanding programming and debugging skills. Strong knowledge of common algorithms and data structures.
Familiarity with one or more stream processing / queuing technologies like Spark Streaming, Kafka, Kinesis, Flink, etc. preferred.
Familiarity and prior experience with Agile / Scrum development methodologies.
Required Experience
Experience with Java and/or Scala, preferably with Spring framework. 3+ years preferred.
Strong experience with SQL and relational databases like PostgreSQL, MySQL, SQL Server or Oracle.
Prior Experience deploying to cloud platforms, preferably Pivotal Cloud Foundry or Cloud Foundry.
Prior experience in Continuous Integration/Continuous Delivery tools and pipelines such as Jenkins, Maven, Gradle, etc.
Education Requirements
Bachelor’s Degree in Computer Science, Computer Engineering or a closely related field.