|
Vince's CSV Parser
|
Contains CSV parser source-adapter declarations and format helpers. More...
#include <algorithm>#include <array>#include <cmath>#include <exception>#include <fstream>#include <limits>#include <memory>#include <unordered_map>#include <unordered_set>#include <utility>#include <vector>#include "../../external/mio.hpp"#include "../basic_csv_parser_simd.hpp"#include "../col_names.hpp"#include "../common.hpp"#include "../csv_exceptions.hpp"#include "../csv_format.hpp"#include "core.hpp"#include "../csv_row.hpp"#include "../speculative/diagnostics.hpp"Go to the source code of this file.
Classes | |
| struct | csv::internals::parser::GuessScore |
| struct | csv::internals::parser::ResolvedFormat |
| class | csv::internals::parser::ICSVParseOrchestrator |
| class | csv::internals::parser::CSVParserDriverBase |
| Abstract base class for source adapters. More... | |
Namespaces | |
| namespace | csv |
| The all encompassing namespace. | |
Functions | |
| size_t | csv::internals::get_bom_skip (csv::string_view data, bool &utf8_bom, CSVError &error) |
| Return leading BOM bytes to skip, reporting unsupported Unicode encodings. | |
| GuessScore | csv::internals::parser::calculate_score (csv::string_view head, const CSVFormat &format) |
| CSVGuessResult | csv::internals::parser::guess_format (csv::string_view head, const std::vector< char > &delims, CSVError &error) |
| Guess the delimiter used by a delimiter-separated values file. | |
| template<typename TStream , csv::enable_if_t< std::is_base_of< std::istream, TStream >::value, int > = 0> | |
| std::string | csv::internals::parser::get_csv_head_stream (TStream &source, CSVError &error) |
| Read the first 500KB from a seekless stream source. | |
| std::pair< std::string, size_t > | csv::internals::parser::get_csv_head_mmap (csv::string_view filename, CSVError &error) |
| Read the first 500KB from a filename using mmap. | |
| std::string | csv::internals::parser::get_csv_head (csv::string_view filename, CSVError &error) |
| Compatibility shim selecting stream on Emscripten and mmap otherwise. | |
| CSVGuessResult | csv::try_guess_format (csv::string_view filename, CSVError &error, const std::vector< char > &delims={ ',', '|', '\t', ';', '^', '~' }) |
| Guess the delimiter, header row, and mode column count of a CSV file. | |
| CSVGuessResult | csv::guess_format (csv::string_view filename, const std::vector< char > &delims={ ',', '|', '\t', ';', '^', '~' }) |
| std::unique_ptr< ICSVParseOrchestrator > | csv::internals::parser::make_csv_parse_orchestrator (const ParseFlagMap &parse_flags, const WhitespaceMap &ws_flags, const CSVFormat &format, size_t source_size, const ColNamesPtr &col_names=nullptr, bool enable_speculative_parallel=true, bool source_size_known=true) |
Contains CSV parser source-adapter declarations and format helpers.
Definition in file driver.hpp.
| GuessScore csv::internals::parser::calculate_score | ( | csv::string_view | head, |
| const CSVFormat & | format | ||
| ) |
Definition at line 11 of file guessing.cpp.
| size_t csv::internals::get_bom_skip | ( | csv::string_view | data, |
| bool & | utf8_bom, | ||
| CSVError & | error | ||
| ) |
Return leading BOM bytes to skip, reporting unsupported Unicode encodings.
BOM handling happens at the source-window level (CSVParseOrchestrator and format guessing); CSVParserCore only ever sees BOM-free bytes.
Definition at line 51 of file driver.cpp.
| std::string csv::internals::parser::get_csv_head | ( | csv::string_view | filename, |
| CSVError & | error | ||
| ) |
Compatibility shim selecting stream on Emscripten and mmap otherwise.
Definition at line 42 of file driver.cpp.
| std::pair< std::string, size_t > csv::internals::parser::get_csv_head_mmap | ( | csv::string_view | filename, |
| CSVError & | error | ||
| ) |
Read the first 500KB from a filename using mmap.
Also returns the total file size so callers avoid a second mmap open.
Definition at line 22 of file driver.cpp.
| std::string csv::internals::parser::get_csv_head_stream | ( | TStream & | source, |
| CSVError & | error | ||
| ) |
Read the first 500KB from a seekless stream source.
Definition at line 55 of file driver.hpp.
| CSVGuessResult csv::internals::parser::guess_format | ( | csv::string_view | head, |
| const std::vector< char > & | delims, | ||
| CSVError & | error | ||
| ) |
Guess the delimiter used by a delimiter-separated values file.
Score each row-length group for every candidate delimiter. The highest score (row_length * count) wins, even if that row length is not the most frequent.
Header detection: If the first row is at least as wide as the winning group, use row 0. Otherwise use that group's first row.
Score each row-length group for every candidate delimiter. The highest score (row_length * count) wins, even if that row length is not the most frequent.
Header detection: If the first row is at least as wide as the winning group, use row 0. Otherwise use that group's first row.
Definition at line 73 of file guessing.cpp.
|
inline |
Definition at line 280 of file orchestrator.hpp.