Job Description
About the Company
Grafana Labs is a remote-first, open-source software company focused on helping organizations monitor, understand, and improve their systems. Its open-source visualization tool, Grafana, has more than 20 million users worldwide. The company also supports over 3,000 organizations, including Bloomberg, JPMorgan Chase, and eBay, with observability solutions through the Grafana LGTM Stack.
Customers can use Grafana Cloud as a fully managed service or the Grafana Enterprise Stack for self-managed deployments. The platform includes Grafana Mimir for metrics, Grafana Loki for logs, and Grafana Tempo for traces.
Grafana Labs combines its open-source roots with a global, collaborative culture built around transparency, trust, autonomy, and meaningful work.
About the Role
Grafana Labs is seeking a Senior AI Engineer to join its Grafana AI teams and develop AI-powered capabilities that help users understand complex observability data and respond to incidents more effectively. The role focuses on building practical AI solutions that automate analysis, reduce manual effort, and help users detect, investigate, and resolve issues across their systems.
You will work closely with product managers, data analysts, designers, and engineering teams to experiment with new ideas, validate them with users, and deliver production-ready features. The position offers significant ownership and the opportunity to help expand AI agents and automated workflows across the observability stack.
This is a remote position open to applicants working in U.S. time zones.
Key Responsibilities
- Develop and deliver high-performance AI features that help users detect, triage, and resolve incidents using observability data and tools.
- Build and refine LLM-powered and agent-based workflows for incident lifecycle management and automated analysis.
- Prototype, test, and validate ideas quickly with real users, using feedback to improve product functionality.
- Collaborate with data analysts, product managers, designers, and engineers to shape AI-driven product features.
- Integrate AI agents with internal tools, alerting systems, runbooks, and developer workflows.
- Use AI and automation tools to improve product capabilities and streamline software development workflows.
- Take ownership of AI solutions from development through delivery, ensuring they are scalable, maintainable, and aligned with real user needs.
- Communicate effectively across teams and contribute to a collaborative, feedback-driven engineering environment.
- Use AI coding assistants for prototyping, test generation, refactoring, documentation, and incident follow-ups, while maintaining strong code review and quality standards.
Required Qualifications
- Experience building production software systems, including backend and/or full-stack applications.
- Experience with large language models (LLMs), prompt engineering, and generative AI-powered applications.
- A proven track record of delivering software that reaches production and is actively used by customers.
- Exposure to cloud-native environments such as AWS, Google Cloud Platform (GCP), or Microsoft Azure.
- Experience using observability tools to understand system behavior and troubleshoot issues.
- A practical approach to AI development, with an emphasis on building reliable, useful solutions rather than experimentation alone.
- Ability to prototype, gather feedback, and iterate quickly.
- Strong ownership and initiative, including the ability to work through ambiguity and define scope with minimal supervision.
- Effective communication and collaboration skills, with an openness to feedback and a solutions-oriented mindset.
Preferred Qualifications
- Experience building or working with AI agent frameworks and multi-agent workflows.
- Familiarity with infrastructure and DevOps tools such as Kubernetes, Docker, and Terraform.
- Knowledge of model fine-tuning techniques.
- Experience building observability tools or related platform capabilities.
Salary and Benefits
The U.S. base salary range for this position is $154,445–$185,334. Actual compensation may vary depending on experience, skills, and level as assessed during the interview process. Compensation ranges and benefits are location-specific, and recruiters will discuss the applicable package with candidates applying from other locations.
The total compensation package may include equity, bonuses where applicable, and other benefits. All roles include Restricted Stock Units (RSUs), giving employees an ownership stake in Grafana Labs.
Additional benefits and workplace offerings include:
- 30 days of annual leave, subject to local legislation, with three days reserved for Grafana Shutdown Days.
- A remote-first work environment and global team culture.
- In-person onboarding to help new employees learn about the company and connect with colleagues.
- Career development opportunities within a growing organization.
- Transparent communication, accessible leadership, and opportunities to contribute to meaningful projects.
- Access to modern AI coding assistants, with a company-funded usage budget and access to advanced models, subject to security guidelines.
Work Arrangement
- Location: Remote
- Eligible Time Zones: United States time zones
- Work Culture: Remote-first, collaborative, and autonomy-focused
- Onboarding: In-person onboarding is part of the employee experience.
Equal Opportunity Statement
Grafana Labs is an equal opportunity employer. The company recruits, trains, compensates, and promotes employees without discrimination based on race, religion, color, national origin, gender, disability, age, veteran status, or other characteristics that make individuals unique. Grafana Labs is committed to building an inclusive and diverse workplace.
Grafana Labs may use AI tools during recruitment to help match information in candidates’ CVs with job requirements. The recruitment team will continue to review applications manually.
Had a job search go strangely, or terribly? Tell us. We publish stories without names. Tell your story →
View Similar Jobs
Similar jobs which you may be interested in. Typically using your existing skillset.