Ai & D

AI & DATA QUALITY ENGINEER

Summary
We are seeking a skilled professional to serve as an AI & Data Quality Engineer within the IT Services sector. This pivotal role focuses on ensuring the integrity, accuracy, and reliability of data pipelines and AI-driven solutions. The successful candidate will bridge the gap between complex data engineering and artificial intelligence, optimizing workflows and maintaining high standards of data quality across our cloud infrastructure. This position is critical for supporting advanced analytics and machine learning initiatives, ensuring that all outputs are derived from trustworthy and well-governed data sources.

Responsibilities

• Design, implement, and maintain robust data quality frameworks using vector databases and big data processing tools like PySpark and Apache Spark.
  
• Manage containerized applications and orchestrate microservices deployments utilizing Docker and Kubernetes environments.
  
• Develop interactive dashboards and reporting solutions using Power BI or comparable business intelligence platforms to visualize key performance metrics.
  
• Integrate and leverage various AI and machine learning frameworks and APIs to enhance data processing capabilities and automate quality checks.
  
• Architect and optimize data workflows and storage solutions on the AWS cloud platform, ensuring scalability and security.
  
• Coordinate data ingestion and transformation processes using Apache Airflow or similar orchestration tools to ensure timely and accurate data delivery.
  

Requirements

Requirements:

• Possess 5.5 to 8 years of professional experience in data engineering, quality assurance, or related technical fields.
  
• Demonstrate proven expertise with vector databases and strong proficiency in PySpark or Apache Spark for large-scale data processing.
  
• Have hands-on experience managing containerization technologies, specifically Docker and Kubernetes.
  
• Show capability in building data visualizations and reports using Power BI or alternative business intelligence tools.
  
• Display a solid understanding of AI/ML frameworks and the ability to work with relevant APIs.
  
• Be well-versed in AWS cloud services and their application in data infrastructure.
  
• Have practical experience implementing orchestration solutions such as Apache Airflow.