Skip to content

Log Description

FastDeploy generates the following log files during deployment. Below is an explanation of each log's purpose. By default, logs are stored in the log directory under the execution path. To specify a custom directory, set the environment variable FD_LOG_DIR.

Inference Service Logs

  • backup_env.*.json : Records environment variables set during instance startup. The number of files matches the number of GPU cards.
  • envlog.* : Logs environment variables set during instance startup. The number of files matches the number of GPU cards.
  • console.log : Records model startup time and other information. This log is also printed to the console.
  • data_processor.log : Logs input/output data encoding and decoding details.
  • fastdeploy.log : Records configuration information during instance startup, as well as request and response details during runtime.
  • workerlog.* : Tracks model loading progress and inference operator errors. Each GPU card has a corresponding file.
  • worker_process.log : Logs engine inference data for each iteration.
  • prefix_cache_manager.log : Records KV Cache logical index allocation for each request and cache hit status.
  • launch_worker.log : Logs model startup information and error messages.
  • gpu_worker.log : Records KV Cache block count information during profiling.
  • gpu_model_runner.log : Contains model details and loading time.

Online Inference Client Logs

  • api_server.log : Logs startup parameters and received request information.

Scheduler Logs

  • scheduler.log : Records scheduler information, including node status and request allocation details.

Speculative Decoding Logs

  • speculate.log : Contains speculative decoding-related information.

Prefix Caching Logs

  • cache_queue_manager.log : Logs startup parameters and received request information.
  • cache_transfer_manager.log : Logs startup parameters and received request information.
  • launch_cache_manager.log : Records cache transfer startup parameters and error messages.

PD Disaggragation Logs

  • cache_messager.log : Logs transmission protocols and messages used by the P instance.
  • splitwise_connector.log : Records data received from P/D instances and connection establishment details.

CudaGraph Logs

  • cudagraph_piecewise_backend.log : Logs CudaGraph startup and error information.