Table of Contents

PHP RFC: CSV Extension (ext/csv)

This proposal builds on Gina Peter Banyard's girgias/csv extension. Gina has not reviewed this RFC, and it should not be read as her endorsement.

Introduction

PHP 8.6 deprecated the CSV methods of SplFileObject without providing a replacement. This RFC proposes a small CSV extension that follows RFC 4180 conventions by default in the Csv\ namespace that fills that gap and, unlike the existing CSV functions, is locale-independent, has no proprietary escape character and supports multibyte delimiters and enclosures. 

<?php
 
function rows(): Generator {
    yield ['id', 'value'];
    for ($i = 1; $i <= 3; $i++) {
        yield [(string) $i, "row $i"];
    }
}
 
$file = sys_get_temp_dir() . '/example.csv';
Csv\collection_to_file($file, rows());
 
foreach (Csv\LazyLaxCollection::createFromFile($file) as $row) {
    echo implode(' | ', $row), "\n";
}
// id | value
// 1 | row 1
// 2 | row 2
// 3 | row 3
 
?>

PHP's built-in CSV tooling (fgetcsv(), fputcsv(), str_getcsv() and the CSV methods of SplFileObject) predates RFC 4180 adoption in the wider ecosystem and carries behaviour that cannot be fixed without breaking compatibility:

PHP 8.6 deprecated SplFileObject::fgetcsv(), SplFileObject::fputcsv(), SplFileObject::setCsvControl() and SplFileObject::getCsvControl() (Deprecations for PHP 8.6) without providing a replacement. Two gaps were raised on the mailing list during that discussion and remain open:

  1. SplFileObject::fputcsv() has no drop-in replacement: the procedural fputcsv() requires a stream resource, and SplFileObject does not expose its underlying stream.
  2. SplFileObject::READ_CSV was not deprecated, yet setCsvControl(), the only way to configure the dialect it uses, was.

This RFC proposes to close those gaps by adding a small, RFC 4180 compliant CSV extension to the core, based on the existing girgias/csv extension, extended with file/stream support.

Proposal

Add a new, always-enabled extension ext/csv providing six functions and one class in the Csv\ namespace. With the default dialect, they follow RFC 4180 conventions: fields containing the delimiter, the enclosure, CR, LF or the EOL sequence are enclosed; enclosures inside enclosed fields are escaped by doubling; there is no escape character. Delimiters, enclosures and EOL sequences may be multibyte, and parsing is binary safe and locale-independent.

Strict parsing

Whatever the dialect, the parser applies the record grammar of RFC 4180 section 2 strictly and throws a ValueError instead of guessing on input that does not follow it:

For accepted dialects with self-overlapping multibyte enclosures, the parser distinguishes a closing enclosure using the following delimiter, EOL sequence or end of input. Dialect combinations for which this remains ambiguous are rejected as described below.

Multibyte and overlapping dialect tokens

Delimiters, enclosures and EOL sequences may contain multiple bytes. Overlapping tokens are supported where they allow unambiguous round-tripping.

Writers enclose a field when it contains CR, LF or a dialect token, or when a token could form across a boundary between the field and an adjacent delimiter or EOL sequence. This includes boundaries between rows.

For example, with delimiter -- and enclosure aa, the row containing - and z is written as aa-aa--z followed by the configured EOL sequence, and reads back unchanged.

Some token combinations cannot represent every row unambiguously, even with enclosing and enclosure doubling. Readers and writers reject these dialects during argument validation with a ValueError. For example, delimiter xy with enclosure xyx is rejected because the delimiter followed by an opening enclosure itself begins with the enclosure token.

Validation also rejects tokens hidden by the parser's token precedence (enclosure, then delimiter, then EOL), enclosures formed by repeating a separator wholly or partially, and overlaps that make a closing enclosure ambiguous. Sharing bytes or having a self-overlap does not by itself make a dialect invalid; for example, delimiter -- with enclosure aa remains supported.

String and row conversion

<?php
 
namespace Csv;
 
function array_to_row(array $fields, string $delimiter = ',', string $enclosure = '"', string $eolSequence = "\r\n"): string {}
 
function row_to_array(string $row, string $delimiter = ',', string $enclosure = '"', string $eolSequence = "\r\n"): array {}
 
?>

array_to_row() formats one row (the inverse of str_getcsv(), which PHP currently lacks); row_to_array() parses exactly one row: a trailing EOL sequence is accepted, any data after it is a ValueError.

Collection conversion

<?php
 
namespace Csv;
 
function collection_to_buffer(iterable $collection, string $delimiter = ',', string $enclosure = '"', string $eolSequence = "\r\n"): string {}
 
function buffer_to_collection(string $buffer, string $delimiter = ',', string $enclosure = '"', string $eolSequence = "\r\n"): array {}
 
function buffer_to_collection_lax(string $buffer, string $delimiter = ',', string $enclosure = '"', string $eolSequence = "\r\n"): array {}
 
?>

collection_to_buffer() accepts any iterable of arrays (including generators) and checks that every row has the same number of fields, throwing a ValueError otherwise, as does buffer_to_collection() when parsing. The _lax variant permits rows of varying width.

File and stream handling

<?php
 
namespace Csv;
 
function collection_to_file(string $file, iterable $collection, string $delimiter = ',', string $enclosure = '"', string $eolSequence = "\r\n"): void {}
 
?>

collection_to_file() writes rows one at a time through the streams layer, so a generator-backed collection of any size is written in constant memory. $file accepts anything the streams layer accepts (paths, php://, compress.zlib://, user wrappers) and is opened with the default stream context (stream_context_set_default()). Failures to open, write or finish writing (e.g. a failing flush of a compressed stream on close) throw; a failure to open carries the reason reported by the wrapper. A row is held for the whole time it is formatted, so a field's __toString() that advances the generator producing the rows is safe. 

<?php
 
namespace Csv;
 
/**
 * @not-serializable
 * @strict-properties
 */
final class LazyLaxCollection implements \IteratorAggregate
{
    private function __construct() {}
 
    public function getIterator(): \InternalIterator {}
 
    public static function createFromBuffer(string $buffer, string $delimiter = ',', string $enclosure = '"', string $eolSequence = "\r\n"): LazyLaxCollection {}
 
    public static function createFromFile(string $file, string $delimiter = ',', string $enclosure = '"', string $eolSequence = "\r\n"): LazyLaxCollection {}
}
 
?>

LazyLaxCollection iterates rows lazily. createFromFile() opens a stream with the default stream context and holds it: rows are read in chunks with an enclosure-aware scanner (an EOL sequence inside an enclosed field does not terminate a row) that resumes where it stopped after each read, so a row is found in linear time however long it is, and the internal window buffer is compacted as rows are consumed, so memory usage is proportional to the longest row rather than to the file. The stream is owned by the object: like the stream of SplFileObject, it remains visible as a resource but fclose() refuses to close it, and it is closed when the object is destroyed (for an object alive until the end of the request, while destructors run, so a user wrapper's stream_close() is still called).

The iteration state belongs to the iterator, not to the collection. Loops over a collection created from a buffer are independent of each other, so nested loops over the same collection see every pair of rows. A collection created from a file reads one stream, so only one loop may iterate it at a time; starting a second one while the first is active throws an Error. Iteration can be restarted (foreach twice), including after an early break, as long as the underlying stream is seekable.

Error handling

Migration from the deprecated SplFileObject API

Deprecated (8.6) Replacement
new SplFileObject($f) + READ_CSV + flags Csv\LazyLaxCollection::createFromFile($f, ...)
SplFileObject::fgetcsv() in a loop foreach (Csv\LazyLaxCollection::createFromFile(...))
SplFileObject::fputcsv() in a loop Csv\collection_to_file($f, $rows, ...)
SplFileObject::setCsvControl() dialect arguments of the calls above
str_getcsv() (no inverse) Csv\row_to_array() / Csv\array_to_row()

Note the deliberate semantic differences from the legacy API: no escape character (in line with the PHP 9.0 default), no [null] for empty lines, no locale dependence, an enclosure inside a non-enclosed field, data after a closing enclosure and an unterminated enclosure are errors rather than literal data, whitespace before an enclosure is not silently dropped, exceptions instead of false, and “\r\n” as the default EOL sequence as specified by RFC 4180. fgetcsv() accepts both CRLF and LF line endings; with ext/csv the EOL sequence is explicit, so files with LF line endings are read by passing “\n”.

Examples

Round trip through a string, with enclosed fields containing the enclosure and a line break: 

<?php
 
$rows = [
    ['id', 'name', 'note'],
    ['1', 'Alice', 'says "hi"'],
    ['2', 'Bob', "multi\nline"],
];
 
$csv = Csv\collection_to_buffer($rows);
echo $csv;
// id,name,note
// 1,Alice,"says ""hi"""
// 2,Bob,"multi
// line"
 
var_dump(Csv\buffer_to_collection($csv) === $rows); // bool(true)
 
?>

Multibyte delimiter, enclosure and EOL sequence: 

<?php
 
$row = Csv\array_to_row(['a', 'b;;c', 'd'], ';;', "''", "\n");
var_dump($row);                                  // string(15) "a;;''b;;c'';;d\n"
var_dump(Csv\row_to_array($row, ';;', "''", "\n")); // ['a', 'b;;c', 'd']
 
?>

Edge cases: strict and lax row widths, and errors: 

<?php
 
try {
    Csv\buffer_to_collection("a,b\r\nc\r\n");
} catch (ValueError $e) {
    echo $e->getMessage(), "\n";
    // Buffer row 2 contains 1 fields compared to 2 fields on previous rows
}
 
var_dump(Csv\buffer_to_collection_lax("a,b\r\nc\r\n")); // [['a', 'b'], ['c']]
 
var_dump(str_getcsv('', escape: '')); // [null]
var_dump(Csv\row_to_array(''));       // ['']
 
try {
    Csv\LazyLaxCollection::createFromFile('/nonexistent/file.csv');
} catch (Error $e) {
    echo $e->getMessage(), "\n";
    // Csv\LazyLaxCollection::createFromFile(): Failed to open stream: No such file or directory
}
 
?>

Input that does not follow RFC 4180 is rejected instead of guessed: 

<?php
 
try {
    Csv\buffer_to_collection("\"a\"b,c\r\n");
} catch (ValueError $e) {
    echo $e->getMessage(), "\n";
    // Closing enclosure sequence must be followed by the delimiter or the EOL sequence
}
 
try {
    Csv\buffer_to_collection("a,b\nc,d\n"); // LF line endings, default EOL "\r\n"
} catch (ValueError $e) {
    echo $e->getMessage(), "\n";
    // A non escaped field must not contain CR or LF characters that are not part of the EOL sequence
}
 
var_dump(Csv\buffer_to_collection("a,b\nc,d\n", eolSequence: "\n")); // [['a', 'b'], ['c', 'd']]
 
?>

Backward Incompatible Changes

No existing built-in function, class or constant is changed. However, the Csv\ top-level namespace would become reserved for this extension, following the policy on namespaces in bundled extensions; userland code declaring symbols in Csv\ may conflict, as with the Random\ and Uri\ namespaces before it.

Proposed PHP Version(s)

Next minor PHP version (the release after PHP 8.6).

RFC Impact

To the Ecosystem

No new syntax. IDEs, language servers and static analysers need stubs for the new functions and class, which can be taken from ext/csv/csv.stub.php. Userland CSV libraries are unaffected.

To Existing Extensions

None. ext/standard and ext/spl are unchanged.

To SAPIs

None beyond a new always-enabled extension.

New Constants

None.

php.ini Defaults

None.

Open Issues

Unaffected PHP Functionality

fgetcsv(), fputcsv(), str_getcsv() and the deprecated SplFileObject CSV methods keep their current behaviour. This RFC neither changes nor deprecates them.

Future Scope

Voting Choices

Please consult the php/policies repository for the current voting guidelines.

Voting has not started. If the RFC proceeds to a vote after the required discussion period, the proposed primary question is:

The voting widget and dates will be added when voting is formally announced.

Patches and Tests

Implementation targeting php-src master: ext/csv, a port of girgias/csv 0.6.0 (BSD-3-Clause, copyright Gina Peter Banyard and Contributors) with the new collection_to_file() and LazyLaxCollection::createFromFile() APIs.

Disclosure: parts of the implementation were written with the assistance of an LLM. The author has reviewed all of it and takes full responsibility for it.

Implementation

References

Rejected Features

Changelog