Options for the CUDA provider that are passed to SessionOptionsAppendExecutionProvider_CUDA_V2. Please note that this struct is similar to OrtCUDAProviderOptions but only to be used internally. Going forward, new cuda provider options are to be supported via this struct and usage of the publicly defined OrtCUDAProviderOptions will be deprecated over time. User can only get the instance of OrtCUDAProviderOptionsV2 via CreateCUDAProviderOptions. More...

#include <cuda_provider_options.h>

Public Attributes
int	device_id = 0

int	has_user_compute_stream = 0

void *	user_compute_stream = nullptr

int	do_copy_in_default_stream = 1

OrtCudnnConvAlgoSearch	cudnn_conv_algo_search = OrtCudnnConvAlgoSearchExhaustive

size_t	gpu_mem_limit = std::numeric_limits<size_t>::max()

onnxruntime::ArenaExtendStrategy	arena_extend_strategy = onnxruntime::ArenaExtendStrategy::kNextPowerOfTwo

OrtArenaCfg *	default_memory_arena_cfg = nullptr

int	cudnn_conv_use_max_workspace = 1

int	enable_cuda_graph = 0

int	cudnn_conv1d_pad_to_nc1d = 0

int	tunable_op_enable = 0

int	tunable_op_tuning_enable = 0

int	tunable_op_max_tuning_duration_ms = 0

int	enable_skip_layer_norm_strict_mode = 0

int	prefer_nhwc = 0

int	use_ep_level_unified_stream = 0

Detailed Description

Options for the CUDA provider that are passed to SessionOptionsAppendExecutionProvider_CUDA_V2. Please note that this struct is similar to OrtCUDAProviderOptions but only to be used internally. Going forward, new cuda provider options are to be supported via this struct and usage of the publicly defined OrtCUDAProviderOptions will be deprecated over time. User can only get the instance of OrtCUDAProviderOptionsV2 via CreateCUDAProviderOptions.

Definition at line 19 of file cuda_provider_options.h.

Member Data Documentation

onnxruntime::ArenaExtendStrategy OrtCUDAProviderOptionsV2::arena_extend_strategy = onnxruntime::ArenaExtendStrategy::kNextPowerOfTwo

Definition at line 27 of file cuda_provider_options.h.

int OrtCUDAProviderOptionsV2::cudnn_conv1d_pad_to_nc1d = 0

Definition at line 32 of file cuda_provider_options.h.

OrtCudnnConvAlgoSearch OrtCUDAProviderOptionsV2::cudnn_conv_algo_search = OrtCudnnConvAlgoSearchExhaustive

Definition at line 24 of file cuda_provider_options.h.

int OrtCUDAProviderOptionsV2::cudnn_conv_use_max_workspace = 1

Definition at line 30 of file cuda_provider_options.h.

OrtArenaCfg* OrtCUDAProviderOptionsV2::default_memory_arena_cfg = nullptr

Definition at line 29 of file cuda_provider_options.h.

int OrtCUDAProviderOptionsV2::device_id = 0

Definition at line 20 of file cuda_provider_options.h.

int OrtCUDAProviderOptionsV2::do_copy_in_default_stream = 1

Definition at line 23 of file cuda_provider_options.h.

int OrtCUDAProviderOptionsV2::enable_cuda_graph = 0

Definition at line 31 of file cuda_provider_options.h.

int OrtCUDAProviderOptionsV2::enable_skip_layer_norm_strict_mode = 0

Definition at line 36 of file cuda_provider_options.h.

size_t OrtCUDAProviderOptionsV2::gpu_mem_limit = std::numeric_limits<size_t>::max()

Definition at line 25 of file cuda_provider_options.h.

int OrtCUDAProviderOptionsV2::has_user_compute_stream = 0

Definition at line 21 of file cuda_provider_options.h.

int OrtCUDAProviderOptionsV2::prefer_nhwc = 0

Definition at line 38 of file cuda_provider_options.h.

int OrtCUDAProviderOptionsV2::tunable_op_enable = 0

Definition at line 33 of file cuda_provider_options.h.

int OrtCUDAProviderOptionsV2::tunable_op_max_tuning_duration_ms = 0

Definition at line 35 of file cuda_provider_options.h.

int OrtCUDAProviderOptionsV2::tunable_op_tuning_enable = 0

Definition at line 34 of file cuda_provider_options.h.

int OrtCUDAProviderOptionsV2::use_ep_level_unified_stream = 0

Definition at line 39 of file cuda_provider_options.h.

void* OrtCUDAProviderOptionsV2::user_compute_stream = nullptr

Definition at line 22 of file cuda_provider_options.h.

The documentation for this struct was generated from the following file:

onnxruntime/core/providers/cuda/cuda_provider_options.h

Public Attributes

Detailed Description

Member Data Documentation