Title:  Trading Engineer - Ag & Trading

Job Purpose and Impact

The Professional, Trading Engineering job maintains and partners to design and develop software applications and systems that drive the trading operations. With limited supervision, this job participates in co-creation and maintenance of sophisticated software applications vital to our trading activities, collaboration with cross functional teams to conceptualize, design and implement new software features that meet user needs and advance our business objectives, including upgrading existing software by writing clean, efficient and scalable code to improve performance and functionality, and identifying and resolving issues promptly to ensure the seamless operation of the software systems. This job also participates in code reviews and shares insights and knowledge to uphold the standards of excellence.

Key Accountabilities

• SOFTWARE DEVELOPMENT: Design, develop, and maintain scalable ETL/ELT pipelines using AWS Glue, Apache Airflow, and Apache Spark/Scala. Build real-time and batch data processing solutions using Apache Kafka and Spark Streaming. Write clean, efficient and maintainable high-performance data transformation code in Scala, customizing solutions to align with trading strategies and regulatory requirements.

• AUTOMATION: Integrate and optimize data workflows with Snowflake for analytics and BI consumption.

Implements continuous integration and deployment pipelines to streamline trading software releases, automates complex tests to validate software reliability under various market conditions and optimizes workflows by automating repetitive tasks to increase operational efficiency.

• COLLABORATION: Collaborates with traders, data scientists,  analysts, and other cross functional teams to gather requirements and deliver needed solutions, ensuring technical solutions aligned with business goals, participates in agile practices like standups and sprint planning for effective communication, shares knowledge through code reviews to drive learning and best practices, and explains technical concepts clearly to nontechnical team members.
• TESTING & DEBUGGING: Develops comprehensive test plans, including unit and integration tests, to validate functionality and performance, applying advanced debugging skills to resolve software defects and maintain system integrity, performing complex testing to ensure the system handles high transaction volumes, and collaborating with quality assurance teams members to ensure software meets standards before deployment.
• CONTINUOUS IMPROVEMENT: Implement data quality, governance, and security best practices across all platforms. Sustains the commitment to improving system performance, scalability and reliability to ensure systems evolve with changing needs, staying current with industry trends and best in class standards, proactively suggesting improvements to development processes and tools, and integrating users and stakeholders feedback to refine software solutions.
• DOCUMENTATION: Reviews and builds technical documentation of complex system architecture and development processes, ensuring clear technical guides and operation manuals to facilitate ease of use, maintaining compliance with regulations and policies, and collaborating to build the knowledge base for delivering onboarding and training new team members.

TECHNICAL SUPPORT: Monitors and troubleshoot data pipelines, live trading systems for optimal performance, ensuring reliability and scalability including addressing complex real time issues promptly, responding quickly to resolve operational problems, minimizing downtime and risk, executing routine maintenance to keep systems secure and efficient, and developing disaster recovery planning to ensure business continuity during system failures or crises

Qualifications

MINIMUM & TYPICAL YEARS OF WORK EXPERIENCE

• Minimum requirement of 3-5 years of work experience in data engineering or big data development.

Technical Skills Required

• Strong programming skills in Scala (or Scala/Python with willingness to work in Scala)
• Hands-on experience with Apache Spark (Data Frames, Spark SQL, Streaming)
• Proficiency in AWS services: Glue, S3, Lambda, EMR, Redshift, IAM
• Experience with Apache Kafka for real-time data ingestion and processing
• Workflow orchestration using Apache Airflow
• Good experience with Snowflake: data loading, transformation, performance tuning
• Experience in Agile/Scrum environments

Preferred Skills

• Worked on large-scale data platform migration initiatives, including legacy-to-cloud transitions and modernization of ETL pipelines

Functional Skills Required

• Hands-on experience in agricultural trading platforms, derivatives pricing, risk-adjusted analytics for futures, options, and hedging strategies          
• Strong understanding of market and credit risk, derivatives trading, risk exposure, and contract lifecycle management
• Proven track record in designing cloud-native data platforms using AWS, Spark, Kafka, Snowflake, and Airflow, with a focus on scalable, secure, and domain-driven solutions