V9.3 - Repository Reprocessing Automation and Enhanced Repository List Management

V9.3 - Repository Reprocessing Automation and Enhanced Repository List Management

Summary

This release focuses on automating repository reprocessing to prevent unnecessary work by leveraging the worked-example-miner-candidates GitHub directory to check the status of repositories. By improving the tracking and management of processed repositories, this version enhances the efficiency of the Worked-Example-Miner (WEM) project.

Key Improvements

Repository Reprocessing Automation

• Introduced a mechanism to check the worked-example-miner-candidates GitHub directory for already processed repositories, preventing reprocessing of previously analyzed candidates.
• Updated the repository JSON list to track repositories that have been processed, ensuring smooth candidate management and eliminating redundant work.

Repository List Enhancements

• Enhanced the handling of repository data in PyDriller/repositories_picker.py to store and update the Candidates_Generated attribute effectively, ensuring accurate tracking of processed repositories.
• Improved the filtering functionality in PyDriller/code_metrics.py to only include repositories that have not yet been processed.

Refactorings and Code Quality

• Refined import statements in multiple files for better code structure and maintainability (PyDriller/metrics_changes.py).
• Renamed JSON attributes for improved clarity and consistency across repository data handling (PyDriller/repositories_picker.py).

Bug Fixes

• Fixed issues with attribute initialization and repository list management to ensure correct marking of processed repositories.

Important Notes

• Users should ensure the updated repository list is properly synced with the worked-example-miner-candidates GitHub directory to take full advantage of the reprocessing automation.
• The enhanced filtering mechanism will ensure that only unprocessed repositories are considered, reducing unnecessary computation.

Full Changelog: v9.2...v9.3

V9.2 - Enhanced Repository Filtering and Improved Submodule Management

Summary

This release introduces key enhancements in repository data management, improved filtering capabilities, and refined handling of submodule dependencies in the Worked-Example-Miner (WEM) project. It emphasizes more efficient processing of repository candidates and better organization of repository attributes.

Key Improvements

Improved Repository Attribute Management

• Added a Candidates_Generated attribute to track repository processing status in PyDriller/repositories_picker.py.
• Updated repository data to set Candidates_Generated to True after processing completion (PyDriller/metrics_changes.py).
• Implemented initialization for the Candidates_Generated attribute to ensure consistency (PyDriller/repositories_picker.py).

Advanced Repository Filtering

• Enhanced repository filtering to include only repositories with Candidates_Generated set to False for more focused candidate generation (PyDriller/code_metrics.py).

Submodule Management Enhancements

• Improved dependency handling by integrating git submodule init and git submodule update into the Makefile dependencies rule for streamlined submodule setup (Makefile).

Refactoring and Code Quality

• Updated import statements across modules for better maintainability (PyDriller/metrics_changes.py).
• Renamed repository JSON attributes for clarity and improved readability (PyDriller/repositories_picker.py).

Bug Fixes

• Resolved inconsistencies with repository attribute naming to align with updated conventions.

Important Notes

• Users are encouraged to update their local repositories to take advantage of the refined candidate processing and submodule management features.
• Ensure all submodules are properly initialized and updated using the enhanced Makefile dependencies rule.

Full Changelog: v9.1...v9.2

V9.1 - Streamlined Environment Setup and Enhanced Metric Thresholds

Summary

This release focuses on streamlining the environment setup process and refining threshold-based metrics functionality in the Worked-Example-Miner (WEM) project. Key updates include improved handling of environment variables, expanded setup automation, and enhanced metric thresholds for more precise analysis of code candidates.

Key Improvements

Enhanced Setup Automation:

• Refined install_requirements.sh to verify package availability and handle critical dependencies like build-essentials, venv, and Java.
• Introduced improved handling and validation of the JAVA_HOME environment variable for a smoother setup process.
• Updated the setup script to include clear and concise messages for better user guidance.

Documentation Improvements:

• Added detailed sections in the README for setting up JAVA_HOME, C/C++ compilers, and Python virtual environments.
• Addressed missing information about required packages, enhancing clarity for new users.

Metric Threshold Features:

• Introduced dictionaries for minimum and maximum metric value thresholds to metrics_changes.py.
• Added maximum threshold functionality for identifying code candidates with metric decreases over time.

Bug Fixes:

• Resolved issues with setting and updating the DEFAULT_REPOSITORIES dictionary across various modules.
• Fixed parameter defaults in key functions like generate_tasks_description and RUN_FUNCTIONS.

Refinements in PyDriller:

• Optimized repository processing functions to return structured dictionaries.
• Improved welcome and process list messages for better user experience and feedback.

Important Notes

• Users are encouraged to update their local repositories to leverage the streamlined setup and enhanced analysis capabilities introduced in this release.
• Ensure all dependencies are met using the updated install_requirements.sh script for optimal functionality.

Full Changelog: v9.0...v9.1

V9.0 - Enhanced Performance, Code Clarity in PyDriller and Setup Automatization.

Summary

This release enhances the foundation laid in version 8.0, with a strong focus on refactoring the PyDriller project to elevate both code clarity and functionality. Numerous functions have been optimized, and redundant code has been eliminated, resulting in improved maintainability and an enriched user experience.

Key Improvements

• Automated Setup Process: Introduced the install_requirements.sh shell script to automate most of the setup process, streamlining the installation for users.
• Enhanced README: Significant additions and improvements to the README file for clearer guidance and documentation.
• CK Metrics Function Enhancements: Refactored CK metrics-related functions for greater efficiency and reduced complexity, facilitating smoother performance.
• Dynamic Repository Processing: Implemented a setup process to determine if a repository requires processing, improving the usability of code_metrics.py.
• Optimized Commit Counting: Enhanced methods for counting commits, leading to better performance when managing repository data.
• Refined Code Churn Calculation: Modularized the functions for calculating code churn and its attributes, resulting in cleaner and more maintainable code.
• Thorough Bug Fixes: Resolved various issues, including accurate metrics writing to files and improved handling of commit data across modules.
• Progress Bar Enhancements: Improved progress bar descriptions in metrics_changes.py for increased clarity and conciseness, enhancing user feedback during execution.
• Streamlined Output Messages: Eliminated duplicate output messages in code_metrics.py, leading to a more organized and user-friendly console output.

Important Notes

• Users are encouraged to update their local repositories to benefit from the enhanced performance, improved code structure, and refined user feedback available in this release.

Full Changelog: View Changelog

V8.0 - REFACTOR: Performance improvement up to 5x in PyDriller/metrics_changes.py.

Release 8.0 - REFACTOR: Performance Improvement Up to 5x in PyDriller/metrics_changes.py

Summary

This release focuses on optimizing the performance of the PyDriller project, achieving up to a 5x improvement in processing speed within the metrics_changes.py module. Various functions have been refactored to enhance efficiency, maintainability, and clarity of the codebase.

Key Improvements

• Performance Optimization: Significant performance enhancements across multiple functions in metrics_changes.py, streamlining data processing and analysis.
• Loop Consolidation: Removed multiple for loops for data generation, integrating them into a single, more efficient loop.
• Code Simplification: Refactored complex functions for better readability and reduced redundancy, improving the overall structure of the code.
• Dynamic Functionality: Implemented features to dynamically generate progress metrics and improve user feedback during processing.

Important Notes

• Users are encouraged to update their local repositories to take advantage of the improved performance and cleaner code structure.

Full Changelog: v7.1...v8.0

V7.1 - FEAT: Adding New DIT, LCOM, LOC AND NOC CK Metrics to PyDriller/metrics_changes.py

Release 7.1 - FEAT: Adding New DIT, LCOM, LOC, and NOC CK Metrics to PyDriller/metrics_changes.py

Summary

This release introduces significant enhancements to the PyDriller project by incorporating new CK metrics: DIT (Depth of Inheritance Tree), LCOM (Lack of Cohesion of Methods), LOC (Lines of Code), and NOC (Number of Children). These additions aim to improve the analysis and evaluation of code quality within software projects.

Features

• New Metrics Added: Implementation of DIT, LCOM, LOC, and NOC metrics.
• Submodule Integration: Added the AutoMetric repository as a submodule, allowing for streamlined updates and functionality.
• Automated Workflow: Established a GitHub workflow to automatically update the AutoMetric submodule hourly, ensuring the latest metrics are always available.
• Improved Functionality: Enhanced repository dictionary management within the repositories_picker.py file for better metric handling.

Fixes

• Addressed various minor issues and typos to enhance code reliability and clarity.

Refactoring

• Cleaned up and optimized existing code to improve readability and maintainability across several modules, particularly in metrics_changes.py and code_metrics.py.

Important Notes

• Ensure to run the update scripts after pulling the latest changes to benefit from the new metrics and workflow improvements.

Full Changelog: v7.0...v7.1

V7.0 - FEAT: Adding the AutoMetric Repository as a Submodule of Worked-Example-Miner

Release 7.0 - Adding the AutoMetric Repository as a Submodule of Worked-Example-Miner

This release introduces the AutoMetric repository as a submodule within the Worked-Example-Miner project, enhancing its functionality by integrating new metrics and improving existing features. Key updates include:

• New Metrics: Added Code Churn and Modified Files to the commit tuple in PyDriller/code_metrics.py.
• RUN_FUNCTIONS Dictionary: Implemented a control mechanism for managing data generation in both code_metrics.py and metrics_changes.py.
• Repository Handling: Enhanced the Setup Repository function to check out repositories at their latest commits and added functionality to retrieve default branch names.
• Code Refactoring: Improved performance by reusing functions, streamlining the commit processing workflow, and fixing various bugs related to commit tuple management and CSV formatting.
• Documentation Improvements: Updated code comments throughout the PyDriller scripts for clarity and accuracy.

This release aims to provide a more robust framework for code quality analysis and metrics extraction, significantly enhancing the user experience and maintainability of the project.

Full Changelog: v6.2...v7.0

V6.2 - FEAT: Adding the Number of Modified Files Metric

Release 6.2 - FEAT: Adding the Number of Modified Files Metric

New Features

• Added the Average Modified Files calculation to enhance repository metrics tracking.
• Implemented filtering of repositories based on Average Modified Files.
• Introduced new CSV files to record Average Modified Files occurrences and percentile intervals.
• Added the Modified Files Count field to the Metrics Track Record, enhancing the metrics evolution analysis.

Improvements

• Enhanced Code Churn computation performance by up to 50x in PyDriller/repositories_picker.py.
• Refactored commit count gathering and sorting to improve code readability and maintainability.
• Improved CSV file handling, including renaming headers and providing verbose output.

Bug Fixes

• Fixed issues with the Code Churn values extraction function call.
• Corrected the id and key extraction logic in the metrics_changes.py to ensure accurate data processing.

Full Changelog: v6.1...v6.2

V6.1 - FEAT: Adding the Code Churn metric and improving its computation by up to 50x in PyDriller/repositories_picker.py

Release 6.1 - FEAT: Adding the Code Churn metric and improving its computation by up to 50x in PyDriller/repositories_picker.py

New Features

• Code Churn Metric: Introduced a new metric to measure code churn, providing deeper insights into code stability and changes over time.
• Repositories Picker: Added a new Makefile rule for selecting repositories to analyze code metrics.
• Code Metrics Analyzer: Enhanced Makefile rule to analyze metrics against repository attributes.
• Metrics Changes Analyzer: New Makefile rule for tracking changes in metrics across repositories.
• Full Path Handling: Implemented full path management for repositories' PDF and JSON files.
• Thread Support: Improved repository setup and filtering functions with multi-threading for better performance.

Improvements

• Performance Boost: Enhanced computation for the Code Churn metric by up to 50 times, increasing the efficiency of metric extraction and analysis.
• Output Enhancements: Updated output formatting for better readability and clarity in various scripts.
• Repository Attributes Updates: Refined methods for updating and handling repository attributes, ensuring accuracy and consistency.

Refactorings

• Cleaned up code by removing redundant function calls and optimizing output handling.
• Standardized naming conventions and organization of constants and functions across scripts.
• Enhanced documentation throughout the codebase, improving maintainability and understanding of functionality.

Fixes

• Resolved issues related to file path conversions and key name discrepancies in metric tracking.
• Corrected verbose output positioning and ensured accurate path constants for repositories.

Documentation

• Updated comments and docstrings to reflect recent changes and improve clarity for future development.

Full Changelog: v6.0...v6.1

V6.0 - Adding a Repositories Picker to Code Metrics Makefile Rule

Release 6.0 - Adding a Repositories Picker to Code Metrics Makefile Rule

Overview

This release introduces a new Repositories Picker feature within the Code Metrics Makefile Rule. The Repositories Picker enhances the ability to select and process repositories, providing a more streamlined and efficient workflow for code analysis.

Commit Summary

Refactoring

• Updated GitIgnore Rules for the Gemini Directory.
• Improved the Generate Zip Files target directories and auxiliary scripts.
• Enhanced the Execution Time Estimate Calculation.
• Fixed a typo in the PyDriller Code Metrics comment.
• Updated the notification sound filepath for auxiliary scripts.
• Refined various constants and functions related to the Repositories Selector.

Features

• Introduced the Start Template for creating the Repositories Selector.
• Added functionality to output the Repositories list as a PDF.
• Implemented sorting of the Repositories list by the number of stars.

Bug Fixes

• Resolved issues with double quotes in F-strings in multiple files.
• Fixed capitalized repository directory names.

Key Changes

• New Functions Added:

  • Fetch and filter repositories
  • Save to JSON and PDF
  • Random selection of repositories

• Improved Logic:

  • Enhanced heuristics for selecting candidates based on refactoring patterns.
  • Updated the terminal output to provide clearer feedback.

• Makefile Rule Update:

  • Added a new rule to run the Repositories Selector seamlessly within the existing workflow.

Notes

This release focuses on improving usability and performance when analyzing repositories. The addition of the Repositories Picker is a significant enhancement that allows users to manage and evaluate multiple repositories more effectively.

For detailed information about each commit, refer to the commit history from v5.0 to v6.0.

Full Changelog: v5.0...v6.0