Description
Sprout Social is looking for a Senior Applied AI/ML Scientist to join its AI, Data, and Intelligence Business Unit and drive advancements to our suite of agentic and ML capabilities that are the foundation of Sprout Intelligence.
Sprout Social empowers businesses worldwide to harness the power of social media. Processing over one billion social messages daily, our platform serves essential insights to over 30,000 brands. We are weaving AI throughout our products, and this role sits at the center of making that AI trustworthy: as we ship agents across our product suite, the hardest and most valuable problem is knowing whether an agent is actually good, and making it better. That is the work of this role.
The Role
Own the development of agents and agentic features alongside our engineering teams.
Scientists define what quality means for our agents, design and calibrate the evaluations and judges that measure it, decide which models to adopt and why, identify and implement improvements to core agentic capabilities, and hold the quality bar as we scale Sprout’s agentic platform. If you are energized by the question "how do we effectively use AI to solve hard problems for our customers”, then this is the role for you.
What You'll Do
- Own end-to-end agent and agentic feature development, including defining requirements, definitions of success, evals, guardrails, the necessary tools; and working with engineers, designers, and PMs to execute.
- Lead model selection, development and validation: rigorously assess whether a model or configuration is genuinely better, and make well-reasoned adoption decisions.
- Design and run rigorous experimentation (A/B testing and other methods) to measure the real impact of AI features on customers.
- Set and evolve the quality bar across agents, and partner with engineers who build and run the agents and the eval infrastructure so your judgment scales across many systems rather than one.
- Identify opportunities to improve the capabilities of our agent platform, ranging from prompt iteration, reusable judges, context window management, tool use, adopting new state of the art models.
- Communicate AI capabilities, limitations, and quality clearly to product, design, engineering, and leadership, so decisions are grounded in evidence.
What You'll Bring
We are looking for a scientist whose strength is evaluation and judgment about AI systems, not primarily model construction. You understand models deeply enough to judge them, and you are excited to own the quality and measurement layer of production AI.
Minimum qualifications:
- 5+ years of applied AI/ML experience, with a strong track record in the evaluation, measurement, and quality of AI/ML or LLM systems that shipped and drove measurable impact.
- Demonstrated experience designing evaluation frameworks, building and calibrating judges or automated evaluators, and establishing their reliability, for ML or LLM/agentic systems.
- Strong judgment in model selection and validation: assessing whether one model or approach is genuinely better than another, and why.
- Rigorous experimentation and statistical skills: A/B testing, power analysis, and sound measurement design.
- Foundational ML and modeling knowledge (Python, common ML frameworks, transformers and embeddings) sufficient to reason about and evaluate the systems you assess. SQL and comfort with large datasets.
- Ability to work closely with product, design, and engineering, and to translate technical quality questions into terms stakeholders can act on.
Preferred qualifications:
- Direct experience evaluating LLM-driven or agentic products in production (prompt quality, judge design, hallucination and quality measurement, tool-calling reliability).
- Experience defining a quality bar or evaluation methodology that other people and teams then built against.
- Rapid prototyping with new AI/ML techniques to assess their applicability.
- Familiarity with eval tooling and observability (e.g. Datadog, Phoenix, or similar) and MLOps practices.
- Experience influencing product roadmaps with evidence about AI quality and feasibility.
- Awareness of AI/ML model security, bias mitigation, and responsible-AI practices.
How You'll Grow
Within 1 month, you’ll plant your roots, including:
- Complete Sprout's New Hire training alongside other new team members.
- Meet your onboarding resources and get familiar with our agents, eval systems, available data, and quality practices.
- Begin meeting with stakeholders across product, engineering, and design to understand the agents in production and how quality is measured today.
- Work with your manager to define your first area of ownership.
Within 3 months, you’ll start hitting your stride by:
- Take ownership of the evaluation and quality of one or more agents, designing or strengthening the evals and judges that measure them.
- Deepen your understanding of Sprout's products, customers, and agents to identify where quality can be improved.
Within 6 months, you’ll be making a clear impact through:
- Own the quality bar for a meaningful part of the agent surface, and drive model-selection and improvement decisions grounded in evidence.
- Contribute to the longer-term roadmap for AI quality and evaluation at Sprout.
- Help refine and guide our evaluation methodology and best practices.
Within 12 months, you’ll make this role your own by:
- Anchor the eval and judgment layer across multiple agents, with your methodology adopted by others.
- Identify new opportunities to raise the quality and capability of our agentic platform.
- Build relationships with business unit leaders, stakeholders, and executives.
- Elevate the technical excellence of the team and our evaluation practices.
- Stay current with state-of-the-art research and bring it to bear.
- Surprise us! Use your unique ideas to improve the team in ways we haven't considered.
Of course what is outlined above is the ideal timeline, but things may shift based on business needs and other projects and tasks could be added at the discretion of your manager.
Our Benefits Program
We’re proud to regularly be recognized for our team, product, and culture. We invest in our team with a comprehensive, competitive benefits program:
- 100% Employer-Paid Health Benefits: We cover 100% of the premiums for you and your eligible dependents, including comprehensive medical, dental (basic & major), vision, life insurance, and disability.
- Generous Paid Time Off: 25 days of vacation annually, plus 5 paid sick days, all public holidays, and additional company-wide Rest & Recharge days.
- Premium Mental Health Support: Full, free access to Modern Health for you and your dependents, including coaching, therapy sessions, and digital wellness resources.
- Annual Lifestyle Stipend: A $950 CAD annual Lifestyle Spending Account to spend on your physical, mental, and financial well-being.
- Remote Work Support: A one-time $550 USD (equivalent) stipend to set up your home office, plus a monthly $50 USD (equivalent) stipend for internet.
- Personalized Financial Wellness: No-cost, confidential access to financial experts through Your Money Line to support your personal financial goals.
- Family & Care Support: Access to subsidized child and eldercare options through Care.com.
- Charitable Giving: A company match for your donations to eligible organizations.
*This list is for informational purposes only. Benefit offerings are discretionary and subject to change and do not constitute a contract or guarantee of benefits.
Our salary ranges reflect the expected earning potential for this role. Individual pay is based on geographic zone, relevant experience, and skills.
Our current hiring range for this role: CA$140,700 – CA$177,900 annually. Offers are made within this range, with opportunities to grow within the broader band based on performance and impact.
We share both our current hiring range and our broader geographic salary bands to provide transparency into our compensation philosophy. This ensures you understand not only your starting potential but also the long-term growth opportunities available as you progress in your role.
The full base pay range for this role is CA$140,700 – CA$211,100.
These ranges were determined by a market-based compensation approach; we used data from trusted third-party compensation sources to set equitable, consistent, and competitive ranges. We also evaluate compensation bi-annually, identify any changes in the market and make adjustments to our ranges and existing employee compensation as needed.
If you require a reasonable accommodation for any part of the interview process or to submit your application, please email us at accommodations@sproutsocial.com. Include the nature of your request and your preferred contact information. We'll do everything we can to support your success during our recruitment process while upholding your privacy. Please note that only inquiries regarding accommodations will receive a response from this email address; other inquiries will not be addressed (e.g., you send your resume but are not requesting an accommodation).
Candidates for this remote work opportunity must be based in either Alberta, British Columbia or Ontario. If you are based in another location within Canada, we aren’t able to hire in your location at this time.
Sprout Social Inc. and its subsidiaries process personal data submitted through your application to assess your qualifications for employment and to inform our hiring decision and, where applicable, for required governmental reporting. For more information, please review Sprout's Global Applicant Privacy Notice.
