Import Issues in 4) Evaluation Notebook - lighteval_evaluate_and_analyse_your_LLM

[dastaiger]

There appears to be an issue in the 4) Evaluation Notebook (when executed in Google Colab)

Issue:
the 3rd Codeblock states:

from lighteval.logging.evaluation_tracker import EvaluationTracker
from lighteval.logging.hierarchical_logger import hlog_warn, htrack
from lighteval.models.model_config import create_model_config
from lighteval.pipeline import EnvConfig, ParallelismManager, Pipeline, PipelineParameters

This throws errors as

ModuleNotFoundError: No module named 'lighteval.logging.hierarchical_logger'

Analysis

from lighteval.logging.hierarchical_logger import hlog_warn, htrack
from lighteval.models.model_config import create_model_config

• Both throw an error. All of these functions are never used in the anyhow Notebook.
• To my knowledge these functions don't even exist in the lighteval lib (0.7 / main)

Proposed Action

Remove the 2 import statement in question.
Notebook runs fine afterwards