Observability Engineer - Remote Job Listing at Harbor Freight Tools in Calabasas, CA (Job ID 1437335)

Description

The Observability Engineer is responsible for designing, building, implementing, and administering the tools and pipelines that give the organization full-stack visibility into its infrastructure, applications, and business services. This role owns the day-to-day operation of the observability platform(s) and the NOC/TOC (Network/Technical Operations Center) function to ensure stability, reliability, availability, and performance of the services the business depends on.
Where traditional monitoring answered, “is it up or down,” this role is expected to build observability that answers “why is it behaving this way” — correlating metrics, logs, traces, and events into actionable insight, and reducing time-to-detect and time-to-resolve across the environment.

Duties and Responsibilities

• Provides expertise and develops observability solutions for:
• Network technology, connections, and cloud networking (SD-WAN, VPC/VNet constructs)
• Directory services (Active Directory, LDAP, Entra ID/Azure AD)
• Messaging and collaboration platforms (Exchange Online, Microsoft 365, Teams)
• eCommerce and customer-facing digital environments
• Remote access and VPN infrastructure
• Wireless, mobile, and endpoint device fleets
• Operating system environments (Windows, Linux)
• Data backup, storage, and disaster recovery systems
• VOIP and unified communications systems
• Virtualized and containerized environments (VMware)
• Public and hybrid cloud infrastructure (AWS, Azure, GCP)
• Critical business services and transaction flows         
• Design, deploy, and maintain observability platforms across metrics, logs, traces, synthetic monitoring, and real user monitoring        
• Own and operate the NOC/TOC function end-to-end: alert intake, triage, escalation, incident coordination, and post-incident review             
• Develop and maintain monitoring/observability standards, naming conventions, and instrumentation guidelines across infrastructure and application teams      
• Build and tune alerting logic to reduce noise and alert fatigue; drive toward actionable, high-signal alerting
• Define, track, and report on SLIs, SLOs, error budgets, and other key observability/operational KPIs  
• Develop dashboards and visualizations that give both technical teams and business stakeholders a clear picture of service health
• Mentor technical staff and cross-functional teams on observability concepts, instrumentation practices, and tooling      
• Develop, document, and continuously improve monitoring and incident response processes and runbooks 
• Troubleshoot and resolve issues within the monitoring/observability stack itself (data gaps, false positives, agent/collector health)           
• Evaluate, pilot, and recommend new observability tools and platforms         
• Partner with Help Desk, Infrastructure, DevOps/SRE, and Application teams to implement monitoring, alerting, and response processes          
• Manage the call notification, paging, and escalation system for alerts 
• Lead capacity planning, performance trending, and fault prediction using historical observability data     
• Integrate observability data with automation/AIOps workflows to enable auto-remediation where appropriate           
• Support security operations by feeding relevant telemetry into SIEM/SOC tooling and collaborating on anomaly detection           
• After-hours work, shift work, and on-call rotation required   

Scope

• Staff supervision and development:  No
• Decision making: 
• Review data and provide recommendations on alert actions, monitoring and responses.
• Travel:  Up to 5%
• Flex Designation:  Anywhere

The anticipated salary range for this position is $87,600– $131,500 depending on location, knowledge, skills, education and experience. This position is also eligible for an annual discretionary bonus. In addition, we offer comprehensive and competitive benefits to Associates (and their families) such as medical, dental, vision, life insurance, short-term and long-term disability. Eligible Associates are able to enroll in our company's 401k plan. Associates will accrue paid time off up to 236 hours per year (inclusive of PTO, floating holidays, and paid holidays). Paid sick time up to 80 hours per year unless otherwise required by law.

Requirements

Education and Experience
Education Requirements

• Bachelors Degree in technical discipline, Computer Science or related field, or equivalent experience. Relevant Certifications are a bonus (Vendor-specific observability certifications, CCNA, AWS/Azure Certifications).

Years of Experience

• 2-4 years experience with network/systems management and observability tools.

Skills

• Hands-on experience with modern observability/monitoring platforms such as Dynatrace, SolarWinds, Zabbix, Datadog, New Relic, Splunk, PRTG, or Nagios
• Experience with log aggregation and analysis platforms i.e. Splunk
• Familiarity with distributed tracing and APM tooling (Dynatrace, Jaeger, Honeycomb, AppDynamics)
• Deep understanding of monitoring protocols and data collection methods: SNMP (v1/v2c/v3), MIB structure, OID notation, syslog, NetFlow/sFlow/IPFIX, WMI, JMX, and RESTful API-based telemetry
• Solid understanding of networking fundamentals: routing, switching, TCP/IP, DNS, and network troubleshooting
• Strong grasp of the “three pillars of observability” (metrics, logs, traces) and how they correlate to reduce MTTD/MTTR
• Strong experience with Windows and Linux systems architecture and administration
• Experience monitoring cloud-native and hybrid environments across AWS, Azure, and/or GCP
• Proficiency in scripting/automation languages (Python, Bash, PowerShell)
• Experience integrating monitoring data with automation, ticketing, and AIOps/anomaly-detection workflows
• Demonstrated experience owning or operating within a NOC/TOC environment across its full lifecycle — detection, triage, escalation, resolution, and postmortem
• Ability to develop quantitative, prioritized action plans to address operational and reliability issues
• Strong incident management skills; familiarity with ITIL or similar frameworks a plus
• Excellent written and oral communication and documentation skills
• Ability to work after hours, shifts, and rotating on call as required

Physical Requirements
Corporate - Remote - General office environment requiring ability to:

• Stand, walk, sit for extended periods of time .
• Speak and listen to others in person and over the phone and video conferencing.
• Use keyboard and read from computer screen and reports.
• The ability to lift up to 15 lbs.

Safety
Must be able to perform this job safely in accordance with standard operating procedures and good manufacturing practices, without endangering the health or safety of self or others.
corporate corporate corporate

 Apply on company website