The Wikimedia Foundation is looking for an experienced Engineering Manager to join the Data Engineering Team. At the Wikimedia Foundation, we operate the world’s largest collaborative project: a top ten website, reaching a billion people globally every month, while incorporating the values of privacy, transparency and community that are so important to our users.
Working closely with other Technology and Product teams, as well as our community of contributors and readers, you will help deliver the next generation of data usage, analysis and access across all Wikimedia projects.
This role is responsible for key data engineering initiatives spanning our work in product analytics, machine learning and search.
Here are some examples of projects we have tackled that you may be excited to lead:
- Releasing public data to the Wikimedia community and the world at large. Our public data offerings are used all over the world by companies and research institutions big and small. A popular example is the Wikipedia Clickstream (a.k.a. Wikipedia’s rabbit hole)
- Deploying an anomaly detection system to monitor Wikipedia accessibility over the world and detect possible outages (or censorship events)
- Implementing and driving adoption of the data catalog
- Migrating Oozie and other bespoke data pipelines to Apache Airflow and Spark 3
- Evangelize privacy conscious ways to compute metrics. Privacy is key to the work we do
We are a fully remote, internationally distributed team. We see each other in person 1-2 times a year during one of our off-sites (the last few have been in places like Berlin, Copenhagen and New York) or Wikimania, the annual international conference for the Wiki community.
Open to candidates located in timezones UTC-8 to UTC+2
You are responsible for:
- Manage a fully remote, globally-distributed team of data engineers
- Coach and develop the team to meet their individual and collective goals
- Help deliver data services to provide accessible, trusted data for insights, research and product reuse
- Partner closely with other teams and departments across the Wikimedia Foundation to deliver data solutions
- Work with product and program management to develop and implement the data engineering roadmap in an agile, iterative manner
- Ensure data is available, reliable, consistent, accessible, secure, and available in a timely manner for external and internal stakeholders and in accordance with our privacy policy
- Balance innovation, evolution and addressing technical debt
- Review and advice in code changes and technical decisions made by team
- Help formalize and improve engineering processes
- Develop an inclusive culture that is innovative and collaborative, both within your team and in the broader organization
Skills and Experience:
- 4+ years of engineering management or technical leadership experience
- Experience working with diverse and remote teams
- Experience supporting multiple teams of data analysts
- Knowledge of best practices in the design of data warehouses and data products
- Deep understanding of challenges of delivering data products at scale
- Expert experience working with Data Engineering tools and frameworks, processes and methodologies
- Experience building data pipelines using tools such as Airflow, Spark, Gobblin, Yarn
- Experience with one or more programming languages:Python, Scala, and Java
- Experience with data query and manipulation languages including SQL
- Experience with data visualization, preferably Superset
- Experience with automated testing and continuous integration
- Solid judgment and ability to prioritize
- Strong customer focus
- Excellent written and verbal communication skills
- BS or MS degree, preferably in Computer Science, or equivalent work experience
Qualities that are important to us:
- Commitment to the mission of the organization and our values
- Commitment to our guiding principles
- Commitment to diversity, equity, and inclusion
- Cross-cultural sensitivity and awareness
- Collaborative working experience
