IBM and Qualcomm Technologies Collaboration Ushers in New Era of Edge AI

A groundbreaking collaboration between IBM and Qualcomm Technologies is poised to redefine edge AI development. The partnership aims to equip developers with a comprehensive platform for creating,deploying,and managing responsible AI solutions directly at the edge,reducing latency and enhancing privacy. By combining IBM’s expertise in enterprise AI and governance with Qualcomm Technologies’ leadership in edge computing, the collaboration promises to streamline the AI development process and unlock new possibilities across various industries, from smart manufacturing to autonomous vehicles.

Revolutionizing edge AI development

The collaboration focuses on addressing the increasing demand for AI solutions that can operate efficiently and securely at the edge, closer to where data is generated.This approach reduces latency, enhances privacy, and enables real-time decision-making, critical for applications in industries such as manufacturing, automotive, and telecommunications.

Key Components of the Collaboration

The partnership leverages several key components to deliver a robust and versatile edge AI platform:

• AI Model Development: Developers can utilize a suite of tools and frameworks to build and train AI models optimized for edge deployment.
• Edge Deployment: The platform facilitates seamless deployment of AI models to Qualcomm Technologies’ edge computing platforms, ensuring optimal performance and efficiency.
• AI Governance: IBM’s expertise in AI governance provides tools and methodologies for monitoring and managing AI models, ensuring fairness, transparency, and compliance with regulatory requirements.

Real-Time Monitoring and Governance

A critical aspect of the collaboration is the emphasis on real-time monitoring of AI models deployed at the edge. This monitoring encompasses key metrics such as fairness,drift,and answer relevance,enabling organizations to track the performance of their Generative AI applications and ensure they are operating as intended.

Conclusion: A New Era of Edge AI

the collaboration between IBM and Qualcomm Technologies represents a significant step forward in edge AI development. By combining their respective strengths, they are providing developers with an unparalleled platform to create, deploy, and manage responsible AI solutions at the edge.

This collaboration not only streamlines the AI development process but also opens up new avenues for innovation across industries. From smart manufacturing to autonomous vehicles,the potential applications are vast and transformative.

Developers and businesses are encouraged to explore this new solution and collaborate to push the boundaries of what is absolutely possible with AI.

Streamlining AI Governance: A Comprehensive Approach to Model Evaluation and Compliance

In today’s rapidly evolving technological landscape, ensuring the responsible and effective deployment of AI models is paramount. Model evaluation, involving running assessments on prompt templates, plays a crucial role in guaranteeing performance and compliance. These evaluations, configurable through wizards or APIs, allow users to select dimensions and metrics, adjust settings like sample sizes and thresholds, and provide test data to map inputs and expected outputs. The Model Risk Governance (MRG) solution further facilitates comprehensive governance across all model types within an association, employing object types like Models, Model Groups, Use cases, and Use Case Reviews to manage compliance, risk ratings, and stakeholder approvals.

The Importance of model Evaluation

Model evaluation is a critical step in the AI lifecycle, ensuring that models perform as was to be expected and adhere to compliance standards. The process involves running thorough assessments on prompt templates to validate their effectiveness and adherence to predefined criteria.

Evaluations can be configured using a user-friendly wizard or through APIs, providing versatility and customization. Users can select specific dimensions and metrics relevant to their models, adjust settings such as sample sizes and thresholds to fine-tune the evaluation process, and provide test data to map inputs and expected outputs. This comprehensive approach ensures that models are rigorously tested and validated before deployment.

The results of these evaluations are readily available in the Evaluations tab, offering valuable insights into metric scores, threshold violations, and visualizations over time. This allows stakeholders to understand model performance and processing efficiency, enabling them to make informed decisions about model deployment and optimization.

Model Risk Governance (MRG) Solution

The Model Risk Governance (MRG) solution offers a comprehensive approach to managing and governing all model types within an organization. It employs a structured framework that includes object types such as Models, Model Groups, Use cases, and Use Case Reviews.

This framework enables organizations to effectively manage compliance, assess risk ratings, and secure stakeholder approvals. The MRG dashboard provides a centralized view of compliance status, validation results, and risk levels, offering a clear and concise overview of the organization’s model landscape.

Furthermore, the MRG solution automates key governance processes through workflows, such as use case approval and model lifecycle management. This automation ensures thorough oversight from the initial development stages to the final deployment, promoting continuous compliance and effective risk management for enterprise AI solutions.

Ensuring Continuous Compliance

continuous compliance is a cornerstone of effective AI governance. The MRG solution is designed to facilitate ongoing monitoring and management of models, ensuring that they consistently meet regulatory requirements and internal policies.

By providing a centralized platform for managing model risk and compliance, the MRG solution enables organizations to proactively identify and address potential issues before they escalate. The automated workflows and comprehensive reporting capabilities streamline the compliance process,reducing the burden on compliance teams and minimizing the risk of non-compliance.

The combination of robust model evaluation and the comprehensive Model Risk Governance (MRG) solution provides organizations with the tools and framework necessary to ensure the responsible and effective deployment of AI models. By prioritizing compliance, risk management, and continuous monitoring, organizations can unlock the full potential of AI while mitigating potential risks.

InstructLab: democratizing Large language Model Contributions Through Simplified AI Development

InstructLab is revolutionizing the landscape of Large Language Model (LLM) contributions by simplifying the development process and fostering community-driven innovation. This novel approach empowers a broader range of individuals to participate in shaping the future of generative AI. By streamlining the process of contributing skills and knowledge to llms, InstructLab aims to democratize access and accelerate advancements in the field.

Key Features of InstructLab

InstructLab offers several key features that simplify the process of contributing to LLMs:

• Taxonomy-Driven Organization: InstructLab utilizes a taxonomy to organize content into cascading directories based on domains and subdomains, ensuring clarity and facilitating easier model tuning.
• Synthetic Data Generation: The platform leverages taxonomy-defined skills or knowledge to generate datasets using an LLM, augmenting the training process.
• Model Training: InstructLab provides customizable options for training models,including multi-phase training to optimize performance on knowledge and skills datasets.
• Request Development and Deployment: Trained and validated models can be published in Qualcomm AI Hub, enabling OEM developers to build custom applications.
• Lifecycle Governance: Watsonx.governance handles lifecycle governance by extending AI best practices from predictive machine learning to generative AI, mitigating risks across models, users, and datasets.

Taxonomy for Skills and Knowledge

InstructLab employs a taxonomy to organize skills and knowledge contributions. This taxonomy is structured as a tree, with domains and subdomains at the end nodes to define specific skills and knowledge areas.

Contributors can define skills, whether grounded or ungrounded, by creating a qna.yaml file. This file includes examples of questions,answers,and optional contexts,along with an attribution.txt file that cites the sources used. For knowledge contributions,InstructLab employs a repository of Markdown files that provide detailed contextual details,linked through the qna.yaml file.

This taxonomy leverages synthetic data alignment for Large Language Models (LLMs), organizing content into cascading directories based on domains and subdomains. This ensures clarity and facilitates easier model tuning.

Further information on the taxonomy can be found in the GitHub taxonomy repository.

Synthetic Data Generation

InstructLab utilizes taxonomy-defined skills or knowledge to generate datasets using a Large Language Model (LLM). This process involves creating synthetic data to augment the training process.

To generate synthetic data, users can run the ilab data generate command. This command can leverage GPU acceleration, if available, to speed up the data generation process. The pipeline is customizable, allowing users to employ option models or endpoints for generation.

The generated dataset is saved in JSONL format within the datasets directory. The files are named skills_train_msgs_*.jsonl and knowledge_train_msgs_*.jsonl, depending on the type of contribution.

Users can confirm the dataset creation by inspecting the output directory using the ls datasets command.

An example command for synthetic data generation is:

ilab data generate 
    --pipeline full 
    --sdg-scale-factor 100 
    --endpoint-url http://localhost:8080/v1 
    --output-dir ./outputdir-watsonxai-endpoint 
    --chunk-word-count 1000 
    --num-cpus 8 
    --model ibm/granite-20b-multilingual

Model Training with InstructLab

Training a model using InstructLab involves several customizable options,depending on the available system resources. The process begins by running the ilab model train command within a Python virtual environment. GPU acceleration considerably enhances training speed.

For advanced workflows, multi-phase training can be employed. In this approach, the model is trained sequentially on knowledge and skills datasets to optimize performance. Once training is complete, the model can be evaluated to select the best checkpoint and tested to compare performance before and after training.

An example command for model training is:

ilab model train 
    --strategy lab-multiphase 
    --phased-phase1-data <knowledge train="" messages="" jsonl=""> 
    --phased-phase2-data <skills train="" messages="" jsonl=""> -y

Application Development and Deployment

After the model is trained and validated on watsonx, it is indeed indeed published in Qualcomm AI Hub, making it accessible for OEM developers to build custom applications.

Qualcomm AI Hub enables developers to:

• Compile and optimize the pre-trained PyTorch model into a format that can be run on a device.
• Submit a profile job to run inference with the compiled model on a real physical device with a Snapdragon or Qualcomm Technologies’ chipset.
• Measure on-device model performance,confirming latency and memory are below required targets and gaining insights on which compute units (NPU,GPU,and CPU) the model layers are running on.
• Verify numerical accuracy of the model with an inference job.

Lifecycle Governance with watsonx.governance

watsonx.governance handles lifecycle governance by extending AI best practices from predictive machine learning to generative AI, mitigating risks across models, users, and datasets. It supports responsible AI through explainability, openness, and compliance with internal and external regulations.

Watsonx.governance enables real-time monitoring of model performance and fairness, and management of lifecycle metadata for models and templates, ensuring compliance. Monitoring metrics include those for RAG, drift, and model performance, while Guardrails restrict outputs of hate speech, aggression, and profanity. The AI risk atlas provides educational guidance on potential risks, supporting organizations in creating robust governance frameworks.

Governance workflows integrate roles from model development to deployment and monitoring, fostering a structured and responsible AI lifecycle.