|
Vince's CSV Parser
|
A standalone header file containing shared code. More...
#include <algorithm>#include <array>#include <cassert>#include <cmath>#include <cstdint>#include <cstdlib>#include <deque>#include <limits>#include <memory>#include <mutex>#include <type_traits>#include <string_view>#include <ranges>Go to the source code of this file.
Namespaces | |
| namespace | csv |
| The all encompassing namespace. | |
Concepts | |
| concept | csv::internals::csv_string_field_range |
| concept | csv::internals::has_to_sv_range |
| concept | csv::internals::csv_row_like |
| concept | csv::internals::csv_write_rows_input_range |
Macros | |
| #define | CSV_ENABLE_THREADS 1 |
| #define | CSV_CONST |
| #define | CSV_PURE |
| #define | CSV_FORCE_INLINE inline |
| #define | CSV_PRIVATE |
| #define | CSV_NON_NULL(...) |
| #define | CSV_MSVC_PUSH_DISABLE(w) |
| #define | CSV_MSVC_POP |
| #define | CSV_EXCEPTIONS_ENABLED 0 |
| #define | CSV_TRY try |
| #define | CSV_CATCH(exception_declaration, ...) catch exception_declaration __VA_ARGS__ |
| #define | CSV_CPLUSPLUS __cplusplus |
| #define | CSV_FALLTHROUGH [[fallthrough]] |
| #define | STATIC_ASSERT(x) static_assert(x, "Assertion failed") |
| #define | CSV_DEBUG_ASSERT(x) assert(x) |
| #define | IF_CONSTEXPR if constexpr |
Expands to if constexpr in C++17 and if otherwise. | |
| #define | CONSTEXPR_VALUE constexpr |
Expands to constexpr in C++17 and const otherwise. | |
| #define | CONSTEXPR_17 constexpr |
| #define | CONSTEXPR_14 inline |
| #define | CONSTEXPR_VALUE_14 const |
| #define | CONSTEXPR constexpr |
Expands to constexpr in decent compilers and inline otherwise. | |
Typedefs | |
| using | csv::string_view = std::string_view |
| The string_view class used by this library. | |
| template<bool B, class T = void> | |
| using | csv::enable_if_t = typename std::enable_if< B, T >::type |
| template<bool B, class T = void> | |
| using | csv::internals::enable_if_t = csv::enable_if_t< B, T > |
| template<typename F , typename... Args> | |
| using | csv::invoke_result_t = typename std::invoke_result< F, Args... >::type |
| template<typename... Ts> | |
| using | csv::void_t = void |
| typedef std::uint32_t | csv::internals::CSVChunkIndex |
| Type used to represent the location of a CSV byte within a larger chunk. | |
| using | csv::internals::ParseFlagMap = std::array< ParseFlags, 256 > |
| An array which maps ASCII chars to a parsing flag. | |
| using | csv::internals::WhitespaceMap = std::array< bool, 256 > |
| An array which maps ASCII chars to a flag indicating if it is whitespace. | |
Enumerations | |
| enum class | csv::internals::ParseFlags { QUOTE_ESCAPE_QUOTE = 0 , QUOTE = 2 | 1 , NOT_SPECIAL = 4 , DELIMITER = 4 | 1 , CARRIAGE_RETURN = 4 | 2 , NEWLINE = 4 | 2 | 1 } |
| An enum used for describing the significance of each character with respect to CSV parsing. More... | |
Variables | |
| const int | csv::internals::PAGE_SIZE = 4096 |
| Size of a memory page in bytes. | |
| constexpr size_t | csv::internals::CSV_CHUNK_SIZE_DEFAULT = 10000000 |
| Default chunk size for lazy-loading large CSV files. | |
| CONSTEXPR_VALUE_14 CSVChunkIndex | csv::internals::CSV_CHUNK_INDEX_MAX = (std::numeric_limits<CSVChunkIndex>::max)() |
| CONSTEXPR_VALUE_14 size_t | csv::internals::CSV_CHUNK_SIZE_MAX = CSV_CHUNK_INDEX_MAX |
| constexpr size_t | csv::internals::CSV_CHUNK_SIZE_FLOOR = 500 * 1024 |
| Minimum supported custom chunk size for CSVFormat::chunk_size(). | |
| constexpr size_t | csv::internals::CSV_HEAD_PREVIEW_SIZE = 500000 |
| Bytes read from the start of a source to resolve its format (header, delimiter guessing) before the first parsing window. | |
| constexpr size_t | csv::internals::CSV_SPECULATIVE_PARALLEL_MIN_BYTES = 50ull * 1024ull * 1024ull |
| Default minimum source size before speculative parallel parsing is considered. | |
| constexpr int | csv::CSV_NOT_FOUND = -1 |
| Integer indicating a requested column wasn't found. | |
| constexpr unsigned | csv::CHAR_OFFSET = std::numeric_limits<char>::is_signed ? 128 : 0 |
| Offset to convert char into array index. | |
A standalone header file containing shared code.
Definition in file common.hpp.
| #define CONSTEXPR constexpr |
Expands to constexpr in decent compilers and inline otherwise.
Intended for functions and methods.
Definition at line 259 of file common.hpp.
| #define CONSTEXPR_14 inline |
Definition at line 212 of file common.hpp.
| #define CONSTEXPR_17 constexpr |
Definition at line 194 of file common.hpp.
| #define CONSTEXPR_VALUE constexpr |
Expands to constexpr in C++17 and const otherwise.
Mainly used for global variables.
Definition at line 192 of file common.hpp.
| #define CONSTEXPR_VALUE_14 const |
Definition at line 213 of file common.hpp.
| #define CSV_CATCH | ( | exception_declaration, | |
| ... | |||
| ) | catch exception_declaration __VA_ARGS__ |
Definition at line 94 of file common.hpp.
| #define CSV_CONST |
Definition at line 63 of file common.hpp.
| #define CSV_CPLUSPLUS __cplusplus |
Definition at line 106 of file common.hpp.
| #define CSV_DEBUG_ASSERT | ( | x | ) | assert(x) |
Definition at line 175 of file common.hpp.
| #define CSV_ENABLE_THREADS 1 |
Definition at line 36 of file common.hpp.
| #define CSV_EXCEPTIONS_ENABLED 0 |
Definition at line 84 of file common.hpp.
| #define CSV_FALLTHROUGH [[fallthrough]] |
Definition at line 128 of file common.hpp.
| #define CSV_FORCE_INLINE inline |
Definition at line 65 of file common.hpp.
| #define CSV_MSVC_POP |
Definition at line 77 of file common.hpp.
| #define CSV_MSVC_PUSH_DISABLE | ( | w | ) |
Definition at line 76 of file common.hpp.
| #define CSV_NON_NULL | ( | ... | ) |
Definition at line 67 of file common.hpp.
| #define CSV_PRIVATE |
Definition at line 66 of file common.hpp.
| #define CSV_PURE |
Definition at line 64 of file common.hpp.
| #define CSV_TRY try |
Definition at line 93 of file common.hpp.
| #define IF_CONSTEXPR if constexpr |
Expands to if constexpr in C++17 and if otherwise.
Definition at line 191 of file common.hpp.
| #define STATIC_ASSERT | ( | x | ) | static_assert(x, "Assertion failed") |
Definition at line 170 of file common.hpp.
| typedef std::uint32_t csv::internals::CSVChunkIndex |
Type used to represent the location of a CSV byte within a larger chunk.
Definition at line 419 of file common.hpp.
| using csv::internals::enable_if_t = typedef csv::enable_if_t<B, T> |
Definition at line 218 of file common.hpp.
| using csv::internals::ParseFlagMap = typedef std::array<ParseFlags, 256> |
An array which maps ASCII chars to a parsing flag.
Definition at line 495 of file common.hpp.
| using csv::internals::WhitespaceMap = typedef std::array<bool, 256> |
An array which maps ASCII chars to a flag indicating if it is whitespace.
Definition at line 498 of file common.hpp.
|
strong |
An enum used for describing the significance of each character with respect to CSV parsing.
Definition at line 456 of file common.hpp.
Returns true if two floating point values are about the same
Definition at line 444 of file common.hpp.
|
constexprnoexcept |
Transform the ParseFlags given the context of whether or not the current field is quote escaped.
Definition at line 467 of file common.hpp.
| csv::internals::STATIC_ASSERT | ( | quote_escape_flag(ParseFlags::NOT_SPECIAL, false) | = =ParseFlags::NOT_SPECIAL | ) |
Optimizations for reducing branching in parsing loop.
Idea: The meaning of all non-quote characters changes depending on whether or not the parser is in a quote-escaped mode (0 or 1)
| CONSTEXPR_VALUE_14 CSVChunkIndex csv::internals::CSV_CHUNK_INDEX_MAX = (std::numeric_limits<CSVChunkIndex>::max)() |
Definition at line 421 of file common.hpp.
|
constexpr |
Default chunk size for lazy-loading large CSV files.
The worker thread reads this many bytes at a time by default (10MB).
CRITICAL INVARIANT: Field boundaries at chunk transitions must be preserved. Bug #280 was caused by fields spanning chunk boundaries being corrupted.
Definition at line 416 of file common.hpp.
|
constexpr |
Minimum supported custom chunk size for CSVFormat::chunk_size().
This lower bound allows memory-constrained environments to reduce parser buffer size while avoiding pathological tiny-buffer overhead.
Definition at line 430 of file common.hpp.
| CONSTEXPR_VALUE_14 size_t csv::internals::CSV_CHUNK_SIZE_MAX = CSV_CHUNK_INDEX_MAX |
Definition at line 423 of file common.hpp.
|
constexpr |
Bytes read from the start of a source to resolve its format (header, delimiter guessing) before the first parsing window.
The mmap backend rewinds this preview to its last complete row, so the boundary fuzzer uses this value to place syntax at mmap's first window.
Definition at line 438 of file common.hpp.
|
constexpr |
Default minimum source size before speculative parallel parsing is considered.
Definition at line 441 of file common.hpp.
| const int csv::internals::PAGE_SIZE = 4096 |
Size of a memory page in bytes.
Used by csv::internals::CSVFieldArray when allocating blocks.
Definition at line 403 of file common.hpp.