Pearson Senior Big Data ETL Engineer in San Francisco, California

Senior Big Data ETL Engineer


The Consolidated Data Store team has an opening for a Senior Software Engineer. Since not every student learns the same way and at the same rate, we build systems that makes studying more efficient and more effective for every student. All Personalized Learning and Analytics starts with data, and our team is expert in getting that data into a place and form that is useful for the business. Adaptive learning systems require student performance and activity data; predictive analytics is driven by data; instructor dashboards display data; the CDS team lives and breathes data. Our systems are cutting edge, big data architected, performant, AWS cloud based, and changing rapidly. Sound interesting? This exciting growth area needs a talented Engineer to join its ranks who would be a bold thinker and innovator. We motivate each other to explore new frontiers in an environment that supports and inspires us to always be better. By pushing the boundaries of technology — and each other to surpass these boundaries — we create seeds of learning that become the catalyst for the world’s innovations, personal and global, large and small.


  • Analyze, design and develop appropriate scheduling and data movement choreography solutions using multi-tenant, highly available software components

  • Actively participates in requirements, architecture, and design meetings

  • Enjoy a healthy technical debate but know how to collaboratively solve problems that move everyone forward

  • Communicate well in person, over email, and via instant messenger within the team and with other development teams to improve quality, advance standards, and handle issues.

  • Work efficiently and collaboratively in an Agile (Scrum or Kanban) team environment

  • Directly and meaningfully impact the experience of students, institutions, and professors

  • Improve and maintain internal monitoring, instrumentation, and overall visibility of our learning systems.

  • Be self-directed and be able to prioritize your own workload.

  • Be willing to learn new skills whether they are directly tied to software development, software testing, server maintenance, or application release and delivery.


As an organization and team, we spend a lot of time together and finding the right fit is as important as technical skills and previous accomplishments. The right person will be:

  • Patient. You will need to work directly with several application teams, product owners, and management levels to develop project strategies and deliver code.

  • Smart. Problem solving is more than just knowing the code; it includes knowing how to approach unique problems.

  • Analytical. You will need to be able to analyze requirements, code, and issues to devise strategies and understand solutions.

  • Flexible. There will be times you’ll need to work long hours, and take your work home with you if/when needed.

  • Eager. Not only willing to learn but quick to accept new challenges and opportunities.

  • Creative. Providing new ways to approach problems and build solutions is what this team is all about.

  • Motivated to Share knowledge - We are building learning teams that take the knowledge gained from each experience they go through and use it to better the team's performance. Our team members need to embrace this and use every success and failure to improve the collective intelligence of the group.


  • The ideal candidate will be detail oriented, self-directed, self-motivated, with a strong capacity for working successfully and flexibly with members across the organization.

  • 7 years of industry-standard ETL development experience

  • Strong understanding of the principles of Data Warehousing, including the use of Fact Tables, Dimension Tables, star schema modeling and snowflake schema modeling.

  • Experience in big data technologies like HDFS, MapReduce, Hive, Pig, Spark, Kineses, Firehose, S3, Redshift

  • Strong SQL and Unix experience

  • Experience and interest working in cloud based environments and managed services preferred

  • Experience with Debian, Jenkins, and automated deployments a plus

  • Experience in Java programming a plus

  • Easy to work with, stays confident and optimistic in face of challenges

  • Strong communications skills and the ability to communicate technical concepts to non-technical people and work with a variety of internal and external development groups


Primary Location: US-CA-San Jose

Other Locations US-CA-San Francisco

Work Locations: US-CA-San Francisco-1255 Battery St 1255 Battery St Suite 220 San Francisco 94111

Job: Engineering

Organization: Technology & Operations

Employee Status: Regular Employee

Job Type: Standard

Shift: Day Job

Job Posting: Apr 20, 2017