AI Platform Engineer

Resilient IT is a security-first managed service provider and cybersecurity compliance firm serving organizations within the Defense Industrial Base. We are an Authorized CMMC Third-Party Assessment Organization (C3PAO), a CMMC Level 2 Certified MSP, and a GTIA Trustmark holder.

We develop and operate our own Governance, Risk, and Compliance platform, along with an internally hosted artificial intelligence platform that is already integrated with portions of the GRC. These technologies support our employees, improve service delivery, strengthen operational consistency, and automate appropriate business and compliance activities.

We are a small but growing organization where employees can take meaningful ownership, solve real problems, and directly influence how the company operates. Our culture is built around security, accountability, continuous improvement, and doing the right thing.

Position Overview

Resilient IT is seeking an experienced, hands-on AI Platform Engineer to assume technical leadership of our existing internally hosted AI platform and advance its integration with our GRC and other internal tools.

This is not a research-only or prompt-engineering position. The AI Engineer will be responsible for making the existing platform reliable, secure, supportable, and increasingly useful in day-to-day operations. This includes managing its technical health, troubleshooting problems, improving model performance, maintaining existing integrations, and developing new AI-assisted capabilities and automations.

Our current platform is built around locally hosted technologies, with supporting components for model routing, retrieval-augmented generation, document processing, vector search, authentication, auditing, and API-based integrations.

The AI Engineer will begin with an established platform and existing GRC integration. The initial responsibility will be to understand the current architecture, validate how its components and integrations operate, resolve outstanding issues, and create a controlled roadmap for improvements.

From there, the role will enhance the platform’s capabilities, expand responsible automation, improve existing workflows, and develop new integrations across ResilientGRC and other approved internal systems.

This Is Not a Typical AI Job

This position requires someone who can move between infrastructure, application development, model behavior, troubleshooting, security, and business process automation.

The right person will not simply deploy models or produce demonstrations. You will take ownership of an operational AI environment, understand what has already been built, identify why components behave the way they do, resolve deficiencies, and improve the platform without disrupting existing functionality.

You will also be expected to recognize when AI should assist a process, when deterministic software is more appropriate, and when a human must remain responsible for the final decision.

What You’ll Do

Assume Ownership of the Existing AI Platform

• Learn and document the current AI architecture, configuration, integrations, workflows, dependencies, and operating procedures.
• Serve as the primary technical owner of Resilient IT’s internally hosted AI platform.
• Manage and optimize Ollama, Qwen models, and the supporting AI application stack.
• Evaluate the current environment for reliability, security, performance, maintainability, and scalability.
• Identify and prioritize existing technical issues, incomplete capabilities, configuration problems, and operational risks.
• Preserve working functionality while implementing controlled improvements.
• Monitor platform availability, model performance, resource utilization, response quality, latency, and system capacity.
• Diagnose and resolve model, infrastructure, application, integration, and data-pipeline issues.
• Manage model installation, configuration, testing, evaluation, upgrades, and controlled deployment.
• Evaluate model configurations, quantization options, context limits, inference settings, and hardware utilization.
• Maintain stable development, testing, and production processes for AI services.
• Establish or improve monitoring, logging, alerting, backup, recovery, and platform health checks.
• Plan for platform scalability, redundancy, and future architectural or hardware improvements.

Improve Model Accuracy and Usefulness

• Review and enhance existing system prompts, structured outputs, reusable workflows, and model-specific configurations.
• Establish evaluation methods for measuring accuracy, completeness, relevance, consistency, hallucination risk, and business value.
• Test model and workflow changes against defined use cases before introducing them into production.
• Develop controlled feedback processes that allow users to report inaccurate or unhelpful results.
• Improve retrieval-augmented generation pipelines, document chunking, metadata, embeddings, vector search, citations, and source grounding.
• Ensure AI-generated answers clearly distinguish authoritative source content from analysis, recommendations, and unresolved factual gaps.
• Reduce unsupported conclusions and prevent the platform from presenting assumptions as verified facts.
• Determine when a larger model, smaller model, specialized workflow, traditional software function, or human review is the appropriate solution.

Enhance the Existing GRC Integration

• Review, maintain, troubleshoot, and enhance the current integration between the AI platform and the GRC.
• Understand the existing APIs, data flows, permissions, AI features, and automation already operating within the platform.
• Preserve current working capabilities while introducing tested and controlled enhancements.
• Resolve reliability, accuracy, performance, usability, and data-handling issues within existing AI-assisted GRC functions.
• Expand AI assistance into additional GRC workflows where there is a defined operational benefit.
• Improve contextual search, guided assistance, document analysis, information extraction, and quality-control capabilities.
• Develop additional secure API-based integrations between the AI platform, GRC, and other approved internal systems.
• Automate repeatable administrative and analytical activities where the resulting output can be validated.
• Ensure all enhancements preserve tenant separation, engagement scope, user permissions, data ownership, auditability, and existing validated functions.
• Make automated actions traceable, reviewable, and reversible when appropriate.
• Maintain clear boundaries between consulting assistance and formal C3PAO assessment activities.
• Ensure AI assists qualified personnel without replacing required assessor judgment, independence, impartiality, or final determinations.

Expand Internal Automation

• Assess existing AI automations and determine where they should be corrected, strengthened, expanded, or retired.
• Work with leadership and department owners to identify additional high-value automation opportunities.
• Translate manual processes into well-defined, controlled, and measurable workflows.
• Integrate the AI platform with approved internal applications, data sources, document repositories, and third-party tools.
• Build agents, services, APIs, background jobs, and workflow automations that perform defined business functions.
• Automate document intake, classification, summarization, comparison, extraction, validation, routing, and reporting where appropriate.
• Develop human-in-the-loop approval processes for sensitive or consequential actions.
• Implement validation rules and exception handling so failed or uncertain automation does not silently produce incorrect results.
• Measure whether each automation saves time, improves consistency, reduces errors, or produces another defined business benefit.
• Maintain an enhancement and automation roadmap based on business priority, technical feasibility, risk, and expected return.

Protect Sensitive Information

• Maintain the platform as a secure, internally controlled environment.
• Apply least privilege, role-based access, authentication, authorization, encryption, logging, and auditability requirements.
• Protect Federal Contract Information, Controlled Unclassified Information, client information, assessment data, and other sensitive records.
• Prevent unauthorized access to information across organizations, tenants, engagements, users, and workflows.
• Ensure prompts, responses, attachments, embeddings, logs, and cached data receive appropriate protection.
• Evaluate new integrations and data flows for privacy, security, compliance, and information-handling risks.
• Identify and remediate prompt injection, data leakage, insecure output handling, excessive agency, and other AI-specific risks.
• Maintain alignment with Resilient IT’s security requirements, internal policies, and applicable customer or regulatory obligations.

Support Users and the Business

• Provide technical support for employees using the AI platform and its GRC capabilities.
• Investigate reported problems and clearly communicate causes, corrective actions, and expected resolution times.
• Develop and maintain user guidance, training materials, operating procedures, and responsible-use standards.
• Help employees understand both the capabilities and limitations of AI-assisted tools.
• Gather user feedback and convert it into prioritized platform improvements.
• Communicate technical decisions in language that business, compliance, and executive stakeholders can understand.
• Participate in planning discussions involving AI capabilities, operational improvements, and product development.