Introducing DNIKit: An Open-Source Python Framework for Analyzing AI Models and Datasets
DNIKit, also known as the Data and Network Introspection toolkit, is a powerful open-source Python framework designed to analyze machine learning models and datasets. Its unique algorithms operate on the intermediate network responses, providing valuable insights into how the network perceives data during different stages of computation.
With DNIKit, you can perform various tasks, including creating comprehensive dataset analysis reports, identifying near-duplicate dataset samples, uncovering rare data samples, annotation errors, or model biases, compressing networks by removing highly correlated neurons, and detecting inactive units in a model.
To enhance the visualization of analyses, DNIKit seamlessly integrates with Symphony, a research platform for creating interactive data science components. Previously published at ACM CHI 2022, Symphony has now been made open-source, allowing multiple stakeholders in AI/ML teams to explore, visualize, and share AI/ML analyses. Symphony is compatible with different data types and models and can be used in various environments, including Jupyter Notebooks and standalone web-based dashboards. It even includes specific components to visualize the results of DNIKit analyses, such as computing dataset familiarity and duplicates.
One noteworthy application of Symphony and DNIKit is interactive, visual dataset analysis, particularly the Dataset Report. This powerful tool enables users to extract model responses and feed them into DNIKit algorithms for deeper analysis. Figure 1 provides a demonstration of this process.
Figure 2 showcases an example of visual dataset exploration using Symphony and DNIKit. The backend computation for widgets like Duplicates, Scatterplot, and Familiarity is powered by DNIKit.
In conclusion, DNIKit is a valuable resource for analyzing machine learning models and datasets. Its integration with Symphony enhances the visualization of analyses, making it easier for AI/ML teams to explore and share their findings. Whether you’re interested in dataset analysis or network introspection, DNIKit is a must-have tool in your AI toolkit. Get started with DNIKit today to unlock new insights into your AI models and datasets.