Repository navigation
feat: php8.3 - #41449
feat: php8.3#41449
Conversation
|
Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes. |
165b80e to
824f405
Compare
2488c08 to
f3b0ca4
Compare
…ion during Log::writeExtra
This test seems to have wrong expectations. As far as I see in the docs, the "null" value is acceptable, and as far as I understand, it should allow all widths and heights. There shouldn't be any reason why requesting a 32x32 preview would fail. The test right below this doesn't have the type set for the config keys. It might work because the code detect the invalid width and height requested, and it might fail early. It should add the "null" type as with the test above. @phil-davis could you confirm? I'm not sure if I'm missing anything. |
…tCreatedPublicLinkUsingTheWebUI()
f88552a to
d92e8a1
Compare
https://github.com/owncloud/core/pull/37166/changes committed these tests. I agree that the test expectations are strange. I expect that if the max width and height are set to null, then that means there is no limit on the size of a requested preview. I suppose that the expectation of a 400 not found error was put there because that is what happened at run-time. Someone should have raised an issue about it, and put the issue number in a comment for these scenario(s). But I don't see any comments. Change the test to expect a "200". |
That scenario seems to fail randomly. I'd suggest to remove the last check ( |
This problem happens for me on my Ubuntu 24.04 laptop. I made a text file containing the Nepali (devanagari script) "सिमसिमे" The large preview image (which I can actually see clearly) is: The latin script transliteration of the correct text is "simsime" - the ligature for each vowel is written "above" the consonant that precedes the vowel. But the rendering into a PNG puts the ligature on the following consonant (except that the last "e" ligature is already at the end, so can't move further right). The resulting latin script transliteration is "smismie". Here is an issue that discusses these sort of problems (the example is not exactly the same!): What do we want to do:
|
|
This is an underlying problem that I can demonstrate locally from the command line on my Ubuntu 24.04 laptop with The "short i" ligature is rendered in the wrong place. I tried various other vowel ligatures, and they seem to render OK. So the other option is that I can change the test so that it avoids using the "short i" ligature, and I can followup in my own time about Devanagari rendering problems in Imagemagick. Google AI tells me: And that sounds like a good analysis of the issue. |
I'd go with this solution. If we make another similar test, I think it should be with a different language. |
|
@phil-davis @jvillafanez I'd be fine to disable this test for the time being - reads like a corner case we can address at a later stage again. I suspect we have bigger tasks ahead. Thank you ;-) |
|
Issue #41480 - I will push a commit to skip this Nepali Devanagari preview test. |
For the test configuration where we use a federated server that is running the same code as the code in this PR, we need to use PHP 8.3. Other federated server configurations test against existing releases of oC10, and for those we need to use PHP 7.4
|
The acceptance test pipelines that use a federated server are currently run in 3 different configurations: (a) and (b) need PHP 7.4 I pushed a commit to do that, let's see. The reason for testing against different federated server versions is that we want a "mixed" federation to work, so that sites can upgrade their oC servers one at a time, and be running mixed oC versions for a period. |
Decide the PHP version needed for the federated server once and pass it to the drone starlark functions that need it.
docs(changelog): add entry for the PHP 8.3 migration (#41449) Signed-off-by: Thomas Müller <1005065+DeepDiver1975@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…iews [10.16] Backport of #41827. That PR carries the per-provider Imagick coder pin from #41834 as well, because #41834 was merged into its branch, so the squash commit on master contains both changes and so does this backport. Bitmap::getResizedPreview() sanitized SVG content before handing it to Imagick::readImageBlob(), but fell back to the ORIGINAL, unsanitized bytes whenever the sanitizer returned an empty string - which it does for any content libxml cannot parse, not only for genuinely malformed SVG. A malformed SVG, or any non-XML payload such as a raw MVG script, therefore reached ImageMagick unsanitized, where an <image xlink:href="MSL:..."> or an MVG "fill 'url(...)'" primitive can execute an MSL script that reads and writes arbitrary files as the web user. getResizedPreview() now rejects content whose libmagic-detected media type is text/*, image/svg*, application/xml or image/x-mvg before calling into Imagick at all, and each provider pins the exact coder it serves instead of letting ImageMagick re-derive the format from the content. Adapted for PHP 7.4, the only version 10.16 supports. Master justifies several of these guards by PHP 8 raising an \Error that escapes catch (\Exception); on 7.4 the same calls only warn, so every such claim was re-derived on the target runtime rather than carried over: - finfo_buffer(false, ...) warns and returns false on 7.4, and detectString() returned that false to the new deny-list, where it collapses to '' and matches no entry. Here the missing guard admitted the content the list exists to reject; it is on PHP 8 that it turns a missing preview into a 500. - the same holds for popen()/fgets()/pclose() in detect() and for fopen(false, ...) in the branch taken without ext-fileinfo. - the (string) cast on $file->getMimeType() is required on 7.4 too: passing null to a userland string-typed parameter is a TypeError on 7.4 as well. Measured, because the surrounding guards are not. 10.16 keeps its own "$stream === false" check and $image->loadFromData($bp); the is_resource() form and the (string) cast on that call are master-only, from #41855 and #41449, and the three-way merge preserved both correctly. The measurements the coder-pin comments rest on were re-taken on owncloudci/php:7.4, this branch's own CI image. It ships the same ImageMagick 6.9.11-60 and Ghostscript 9.55.0 as the 8.3 image and every figure reproduced: plain PostScript and EPSF-branded content both render 612x792 unpinned, pinned EPS and pinned PS alike; a %!PS-Adobe payload read unpinned reaches the PS coder at 612x792 while the TTF pin gives 800x480. That image also registers no SVG coder and no HEIF coder distinct from HEIC - which is precisely what PDFTest's old SVG-based guard got wrong and what Heic's single HEIC pin is there for. Verified in owncloudci/php:7.4: tests/lib/Preview/ plus tests/lib/Files/Type/DetectionTest.php at 68 tests / 207 assertions, against 41 tests / 123 assertions before, with 12 environment skips (Movie, Office, SVG). The PDF cases run here for the first time. php -l clean under 7.4 on all 21 changed files. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Thomas Müller <323649642+oc-tmueller@users.noreply.github.com>
…iews [10.16] (OC10-164) (#41863) * fix: prevent arbitrary file write via unsanitized SVG/MVG bitmap previews [10.16] Backport of #41827. That PR carries the per-provider Imagick coder pin from #41834 as well, because #41834 was merged into its branch, so the squash commit on master contains both changes and so does this backport. Bitmap::getResizedPreview() sanitized SVG content before handing it to Imagick::readImageBlob(), but fell back to the ORIGINAL, unsanitized bytes whenever the sanitizer returned an empty string - which it does for any content libxml cannot parse, not only for genuinely malformed SVG. A malformed SVG, or any non-XML payload such as a raw MVG script, therefore reached ImageMagick unsanitized, where an <image xlink:href="MSL:..."> or an MVG "fill 'url(...)'" primitive can execute an MSL script that reads and writes arbitrary files as the web user. getResizedPreview() now rejects content whose libmagic-detected media type is text/*, image/svg*, application/xml or image/x-mvg before calling into Imagick at all, and each provider pins the exact coder it serves instead of letting ImageMagick re-derive the format from the content. Adapted for PHP 7.4, the only version 10.16 supports. Master justifies several of these guards by PHP 8 raising an \Error that escapes catch (\Exception); on 7.4 the same calls only warn, so every such claim was re-derived on the target runtime rather than carried over: - finfo_buffer(false, ...) warns and returns false on 7.4, and detectString() returned that false to the new deny-list, where it collapses to '' and matches no entry. Here the missing guard admitted the content the list exists to reject; it is on PHP 8 that it turns a missing preview into a 500. - the same holds for popen()/fgets()/pclose() in detect() and for fopen(false, ...) in the branch taken without ext-fileinfo. - the (string) cast on $file->getMimeType() is required on 7.4 too: passing null to a userland string-typed parameter is a TypeError on 7.4 as well. Measured, because the surrounding guards are not. 10.16 keeps its own "$stream === false" check and $image->loadFromData($bp); the is_resource() form and the (string) cast on that call are master-only, from #41855 and #41449, and the three-way merge preserved both correctly. The measurements the coder-pin comments rest on were re-taken on owncloudci/php:7.4, this branch's own CI image. It ships the same ImageMagick 6.9.11-60 and Ghostscript 9.55.0 as the 8.3 image and every figure reproduced: plain PostScript and EPSF-branded content both render 612x792 unpinned, pinned EPS and pinned PS alike; a %!PS-Adobe payload read unpinned reaches the PS coder at 612x792 while the TTF pin gives 800x480. That image also registers no SVG coder and no HEIF coder distinct from HEIC - which is precisely what PDFTest's old SVG-based guard got wrong and what Heic's single HEIC pin is there for. Verified in owncloudci/php:7.4: tests/lib/Preview/ plus tests/lib/Files/Type/DetectionTest.php at 68 tests / 207 assertions, against 41 tests / 123 assertions before, with 12 environment skips (Movie, Office, SVG). The PDF cases run here for the first time. php -l clean under 7.4 on all 21 changed files. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Thomas Müller <323649642+oc-tmueller@users.noreply.github.com> * test: make BitmapStreamTest survive the OC10-164 coder pin [10.16] Backport of #41838. Without it the previous commit leaves this file red: testClosesTheStreamOnSuccess fed a PNG through the Photoshop provider, and the PSD pin refuses exactly that mismatch. The fixture is replaced by a PSD Imagick writes itself, so the case needs no fixture and cannot skip - it must not, since the success-path fclose() assertion is the whole subject of the file. The undecodable payload becomes bytes no coder claims, with a control assertion mirroring isDangerousToDecode(): both existing assertions are satisfied by any early return, so a build whose libmagic read the old payload as text/* would have had it refused at the new mime gate, stayed green, and silently stopped covering the decode. Measured on owncloudci/php:7.4 as application/octet-stream, which does reach the decode. Diverges from master in three places, all because this branch is PHP 7.4: - the unopenable-file case keeps its 10.16 shape, asserting that no warning is emitted. Master asserts the return value, which cannot fail here: stream_get_contents(false) only warns on 7.4 and the result is false either way, so the returned value cannot tell an unopenable file from an undecodable one. Only the warning can. - master's note that an unstubbed getMimeType() mock yields a TypeError does not hold on this branch: getThumbnail() casts the value, so an unstubbed mock gives ''. That is worse rather than better for a test - seven of the eight providers answer '' with the same constant they answer anything with, so the case would pass while proving nothing about which coder ran. Font is the one provider that branches on the mime type. The comment says that instead. - owncloudci/php:7.4 rather than :8.3 named as the build with no SVG renderer, confirmed on it. Verified in owncloudci/php:7.4: 3 tests / 7 assertions / 0 failures. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Thomas Müller <323649642+oc-tmueller@users.noreply.github.com> * fix: report a preview file that cannot be opened without logging noise [10.16] Backport of #41855, folded into this backport at the maintainer's request rather than opened as its own 10.16 PR. SVG::getThumbnail() read the handle from $file->fopen('r') without checking it, and released it only on the success path. Bitmap::getThumbnail() checked for false but not for null. Both now use !is_resource(), and the SVG provider closes the handle in a finally so a read that throws from the wrapper stack - the encryption module does, on a missing or damaged key - cannot hold the descriptor and the view's shared lock for the rest of the request. Reworded and re-tested for PHP 7.4, where the consequence differs from master's: - master's changelog and comments say an unopened file made the request fail with a server error. On 7.4 it does not. stream_get_contents() warns and hands on false, the prefix check turns that into a bare XML declaration, and Imagick rejects it - so the preview already degraded to a media type icon, after two misleading log lines. It is on PHP 8 that those calls raise a TypeError, an \Error that escapes the catch (\Exception) as a 500. The changelog now describes the logging noise, which is what this fix removes here. - SVGStreamTest's unopenable case therefore asserts that no warning is emitted, not that the return value is false: master's assertFalse() passes with the guard reverted on this runtime, so it could never go red. Renamed accordingly, and the handler honours error_reporting() so that diagnostics the code under test silences with @ cannot fail it, the same line OC\Log\ErrorHandler::onError() draws. Verified red before green - see the PR description for the counts. - BitmapStreamTest's equivalent case keeps its existing 7.4 assertion and gains #41855's data provider, so the null handle is covered here too. Verified in owncloudci/php:7.4: tests/lib/Preview/ plus tests/lib/Files/Type/DetectionTest.php at 72 tests / 219 assertions / 0 failures, 12 environment skips (Movie, Office, SVG - this image registers no SVG coder). php -l clean under 7.4. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Thomas Müller <323649642+oc-tmueller@users.noreply.github.com> * docs: point the changelog entries at the reopened backport PR The backport PR was reopened as #41863, because #41828 had been opened under the wrong GitHub account and a PR's author cannot be changed. The three changelog entries linked #41828, which is now closed, so they move to #41863. The master PR links are unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Thomas Müller <323649642+oc-tmueller@users.noreply.github.com> --------- Signed-off-by: Thomas Müller <323649642+oc-tmueller@users.noreply.github.com> Co-authored-by: Thomas Müller <323649642+oc-tmueller@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

Description
Related Issue
Motivation and Context
How Has This Been Tested?
Screenshots (if appropriate):
Types of changes
Checklist: