Big Data Engineer
Big Data Engineer
Federal Hill Consulting has partnered with a large financial firm to build a team of Big Data Engineers with a mission to protect the US Securities Capital Markets. We have a portfolio of ‘surveillance patterns’ that look for manipulative and non-compliant behavior in the database of all transactions that occur in the stock market. The database itself consists of tens of billions of records per day. Engineers work with these large volumes of data using state-of-the-art and industry standard technologies, all of which are wholly operated in a cloud computing environment.
In order to operate with ever improving effectiveness, the team operates in a rich culture of performance and innovation. Collaboration within the team and with business stakeholders is frictionless, with significant independence offered in order to experiment with better ways of conducting technology for business value. Technological and career growth opportunities are a natural and every day part of the working environment.
Please read further to get a better feel for the role and the skills that lead to success in this team.
- Analyze system requirements and design responsive algorithms and solutions
- Use big data and cloud technologies to produce production quality code
- Engage in performance tuning and scalability engineering
- Work with team, peers and management to identify objectives and set priorities
- Perform related SDLC engineering activities like sprint planning and estimation
- Work effectively in small agile teams
- Provide creative solutions to problems
- Identify opportunities for improvement and execute
- Experience with cloud based Big Data technologies
- Proficiency in Hive / Spark SQL
- Experience with Spark
- Experience with one or more programming languages like Scala, Python, and/or Java
- Ability to push the frontier of technology and independently pursue better alternatives