Skip to content

[QA] transfer-ownership tests started failing #276

Closed

Description

@phil-davis

For example:
https://drone.owncloud.com/owncloud/encryption/1887/149/11 2021-05-21am

Feature: transfer-ownership

  @smokeTest @skipOnEncryptionType:user-keys
  Scenario: transferring ownership of a file                                                                                                                            # /var/www/owncloud/testrunner/tests/acceptance/features/cliMain/transfer-ownership.feature:6
    Given user "Alice" has been created with default attributes and small skeleton files                                                                                # FeatureContext::userHasBeenCreatedWithDefaultAttributes()
    And user "Brian" has been created with default attributes and small skeleton files                                                                                  # FeatureContext::userHasBeenCreatedWithDefaultAttributes()
    And user "Alice" has uploaded file "filesForUpload/textfile.txt" to "/somefile.txt"                                                                                 # FeatureContext::userHasUploadedAFileTo()
    When the administrator transfers ownership from "Alice" to "Brian" using the occ command                                                                            # TransferOwnershipContext::theAdministratorTransfersOwnershipFromToUsingTheOccCommand()
    Then the command should have been successful                                                                                                                        # OccContext::theCommandShouldHaveBeenSuccessful()
    And the downloaded content when downloading file "/somefile.txt" for user "Brian" with range "bytes=0-6" from the last received transfer folder should be "This is" # TransferOwnershipContext::downloadedContentWhenDownloadingForUserWithRangeShouldBe()
      The downloaded content was expected to be 'This is', but actually is '<?xml v'. HTTP status was 500
      Failed asserting that two strings are equal.
      --- Expected
      +++ Actual
      @@ @@
      -'This is'
      +'<?xml v'

...

runsh: Total unexpected failed scenarios throughout the test run:
cliMain/transfer-ownership.feature:6
cliMain/transfer-ownership.feature:15
cliMain/transfer-ownership.feature:29
cliMain/transfer-ownership.feature:39
cliMain/transfer-ownership.feature:50
cliMain/transfer-ownership.feature:62
cliMain/transfer-ownership.feature:74

And the same each night:
https://drone.owncloud.com/owncloud/encryption/1890/149/11 2021-05-22
https://drone.owncloud.com/owncloud/encryption/1891/149/11 2021-05-23
https://drone.owncloud.com/owncloud/encryption/1892/149/11 2021-05-24
https://drone.owncloud.com/owncloud/encryption/1893/149/11 2021-05-25

The test pipeline uses mySQL 8.0

Activity

  1. phil-davis commented on May 25, 2021

    @phil-davis
    ContributorAuthor

    Note: also see owncloud/core#38770 - update-testing CI also started failing the same night. Maybe related? But they could be caused by different PRs that were merged to core on 2021-05-20.

  2. phil-davis commented on May 25, 2021

    @phil-davis
    ContributorAuthor

    Note encryption PR #273 was merged on 2021-05-19 and the transfer-ownership tests passed in the nightly that ran on the morning of 2021-05-20. So I don't think that encryption PR is the problem.

  3. phil-davis commented on May 25, 2021

    @phil-davis
    ContributorAuthor

    This is an example from owncloud.log:
    https://drone.owncloud.com/owncloud/encryption/1893/149/8

    {"reqId":"hstrlQbRBvcg7vqgNmMC","level":4,"time":"2021-05-25T00:22:16+00:00","remoteAddr":"192.168.6.4","user":"Brian","app":"webdav","method":"GET","url":"\/https\/github.com/remote.php\/webdav\/transferred%20from%20Alice%20on%2020210525_122209\/somefile.txt","message":"Exception: Bad Signature: {\"Exception\":\"OC\\\\HintException\",\"Message\":\"Bad Signature\",\"Code\":0,\"Trace\":
    \"#0 \\\/var\\\/www\\\/owncloud\\\/server\\\/apps\\\/encryption\\\/lib\\\/Crypto\\\/Crypt.php(481): OCA\\\\Encryption\\\\Crypto\\\\Crypt->checkSignature('(\\\\x9E-}\\\\xAE`G\\\\x9F\\\\x90l\\\\xDC:\\\\xA4\\\\xF9M...', '0\\\\xF0j\\\\xDA\\\\xC0\\\\xE4\\\\xCF[\\\\x07\\\/\\\\x9F\\\\x06D\\\\xDB\\\\x8B...', '60febb55d6767ad...')
    #1 \\\/var\\\/www\\\/owncloud\\\/server\\\/apps\\\/encryption\\\/lib\\\/Crypto\\\/Encryption.php(373): OCA\\\\Encryption\\\\Crypto\\\\Crypt->symmetricDecryptFileContent('(\\\\x9E-}\\\\xAE`G\\\\x9F\\\\x90l\\\\xDC:\\\\xA4\\\\xF9M...', '0\\\\xF0j\\\\xDA\\\\xC0\\\\xE4\\\\xCF[\\\\x07\\\/\\\\x9F\\\\x06D\\\\xDB\\\\x8B...', 'AES-256-CTR', 0, 0, true)
    #2 \\\/var\\\/www\\\/owncloud\\\/server\\\/lib\\\/private\\\/Files\\\/Stream\\\/Encryption.php(473): OCA\\\\Encryption\\\\Crypto\\\\Encryption->decrypt(*** sensitive parameters replaced ***)
    #3 \\\/var\\\/www\\\/owncloud\\\/server\\\/lib\\\/private\\\/Files\\\/Stream\\\/Encryption.php(289): OC\\\\Files\\\\Stream\\\\Encryption->readCache()
    #4 [internal function]: OC\\\\Files\\\\Stream\\\\Encryption->stream_read(8192)
    #5 \\\/var\\\/www\\\/owncloud\\\/server\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/streams\\\/src\\\/Wrapper.php(55): fread(Resource id #63, 8192)
    #6 \\\/var\\\/www\\\/owncloud\\\/server\\\/lib\\\/private\\\/Files\\\/Stream\\\/Checksum.php(108): Icewind\\\\Streams\\\\Wrapper->stream_read(8192)
    #7 [internal function]: OC\\\\Files\\\\Stream\\\\Checksum->stream_read(8192)
    #8 \\\/var\\\/www\\\/owncloud\\\/server\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/streams\\\/src\\\/Wrapper.php(55): fread(Resource id #66, 8192)
    #9 \\\/var\\\/www\\\/owncloud\\\/server\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/streams\\\/src\\\/CallbackWrapper.php(96): Icewind\\\\Streams\\\\Wrapper->stream_read(8192)
    #10 [internal function]: Icewind\\\\Streams\\\\CallbackWrapper->stream_read(8192)
    #11 \\\/var\\\/www\\\/owncloud\\\/server\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/streams\\\/src\\\/Wrapper.php(55): fread(Resource id #70, 8192)
    #12 \\\/var\\\/www\\\/owncloud\\\/server\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/streams\\\/src\\\/CallbackWrapper.php(96): Icewind\\\\Streams\\\\Wrapper->stream_read(8192)
    #13 [internal function]: Icewind\\\\Streams\\\\CallbackWrapper->stream_read(8192)
    #14 \\\/var\\\/www\\\/owncloud\\\/server\\\/lib\\\/composer\\\/sabre\\\/http\\\/lib\\\/Sapi.php(112): stream_copy_to_stream(Resource id #73, Resource id #75, 7)
    #15 \\\/var\\\/www\\\/owncloud\\\/server\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(490): Sabre\\\\HTTP\\\\Sapi::sendResponse(Object(Sabre\\\\HTTP\\\\Response))
    #16 \\\/var\\\/www\\\/owncloud\\\/server\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(253): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))
    #17 \\\/var\\\/www\\\/owncloud\\\/server\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(321): Sabre\\\\DAV\\\\Server->start()
    #18 \\\/var\\\/www\\\/owncloud\\\/server\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/webdav.php(66): Sabre\\\\DAV\\\\Server->exec()
    #19 \\\/var\\\/www\\\/owncloud\\\/server\\\/remote.php(165): require_once('\\\/var\\\/www\\\/ownclo...')
    #20 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/server\\\/apps\\\/encryption\\\/lib\\\/Crypto\\\/Crypt.php\",\"Line\":504}"}
    
  4. phil-davis commented on May 25, 2021

    @phil-davis
    ContributorAuthor

    There is an owncloud.log traceback above.
    @micbar which developer would be best to look into this?

  5. micbar commented on May 25, 2021

    @micbar
    Contributor

    @karakayasemi Can you take a look?

  6. IljaN commented on May 25, 2021

    @IljaN
    Contributor

    I am also already looking in to this since it is blocking me. But a second pair of eyes won't hurt I guess.

  7. IljaN commented on May 25, 2021

    @IljaN
    Contributor

    In my local environment a different set of tests fails when running ~/c/c/a/encryption:λ make test-acceptance-cli (inside encryption app)

        When the administrator successfully recreates the encryption masterkey using the occ command                                  # EncryptionContext::recreateMasterKeyUsingOccCommand()
          The command was not successful, exit code was 3.
          stdOut was: 'Master key is not enabled.
          
          '
          stdErr was: ''
           (Exception)
    
    --- Failed scenarios:
    
        /home/ilja/code/core/apps/encryption/tests/acceptance/features/cliEncryption/recreatemasterkey.feature:4
        /home/ilja/code/core/apps/encryption/tests/acceptance/features/cliEncryption/recreatemasterkey.feature:9
    

    Not sure if related to my env.

  8. phil-davis commented on May 25, 2021

    @phil-davis
    ContributorAuthor

    @IljaN the tests expect that the encryption app is already enabled, and the desired encryption setting (master or user keys) has been selected.

  9. IljaN commented on May 25, 2021

    @IljaN
    Contributor

    Hmhh Tests seem to pass locally. Latest core master, trying previouse encryption versions, php7.4
    Trying in 7.2 + mysql

  10. self-assigned this
    on May 26, 2021
  11. IljaN commented on May 26, 2021

    @IljaN
    Contributor

    I can reproduce this now.

  12. IljaN commented on May 26, 2021

    @IljaN
    Contributor

    Bisect first bad commit: owncloud/core@08f84d8

  13. IljaN commented on May 26, 2021

    @IljaN
    Contributor

    Tested by reverting => Tests pass 🟢
    Environment: php7.2 + mariadb

    FYI @jvillafanez

  14. IljaN commented on May 26, 2021

    @IljaN
    Contributor
  15. jvillafanez commented on May 26, 2021

    @jvillafanez
    Member

    Not sure if it will be helpful, but below it's a extract of the filecache before and after the transfer (encryption is disabled):

    MariaDB [owncloud]> select * from oc_filecache where name = 'tenor.gif';
    +--------+---------+-----------------+----------------------------------+--------+-----------+----------+----------+--------+------------+---------------+-----------+------------------+----------------------------------+-------------+-----------------------------------------------------------------------------------------------------+
    | fileid | storage | path            | path_hash                        | parent | name      | mimetype | mimepart | size   | mtime      | storage_mtime | encrypted | unencrypted_size | etag                             | permissions | checksum                                                                                            |
    +--------+---------+-----------------+----------------------------------+--------+-----------+----------+----------+--------+------------+---------------+-----------+------------------+----------------------------------+-------------+-----------------------------------------------------------------------------------------------------+
    |     11 |       1 | files/tenor.gif | 3fa3429aa7e697186c0293aec6eff8eb |      3 | tenor.gif |        7 |        5 | 306793 | 1612273529 |    1612273529 |         0 |                0 | 5cd5ad12e337884b4a731cb2106f77f7 |          27 | SHA1:a4572c2626f8e71fb4cf3b2df16212a06cd164a6 MD5:7b1f9d1688f5df5ed4706c34dafacfa7 ADLER32:89f11464 |
    +--------+---------+-----------------+----------------------------------+--------+-----------+----------+----------+--------+------------+---------------+-----------+------------------+----------------------------------+-------------+-----------------------------------------------------------------------------------------------------+
    1 row in set (0.001 sec)
    
    MariaDB [owncloud]> select * from oc_filecache where name = 'tenor.gif';
    +--------+---------+-----------------------------------------------------------+----------------------------------+--------+-----------+----------+----------+--------+------------+---------------+-----------+------------------+----------------------------------+-------------+-----------------------------------------------------------------------------------------------------+
    | fileid | storage | path                                                      | path_hash                        | parent | name      | mimetype | mimepart | size   | mtime      | storage_mtime | encrypted | unencrypted_size | etag                             | permissions | checksum                                                                                            |
    +--------+---------+-----------------------------------------------------------+----------------------------------+--------+-----------+----------+----------+--------+------------+---------------+-----------+------------------+----------------------------------+-------------+-----------------------------------------------------------------------------------------------------+
    |     11 |       3 | files/transferred from admin on 20210526_091542/tenor.gif | f24455ec205f27299f3ad1c9c51d67ff |      3 | tenor.gif |        7 |        5 | 306793 | 1612273529 |    1612273529 |         0 |                0 | 5cd5ad12e337884b4a731cb2106f77f7 |          27 | SHA1:a4572c2626f8e71fb4cf3b2df16212a06cd164a6 MD5:7b1f9d1688f5df5ed4706c34dafacfa7 ADLER32:89f11464 |
    |     20 |       3 | files/transferred from admin on 20210526_091542/tenor.gif | e40f0aab7f60a24199992985c766ebc0 |      3 | tenor.gif |        7 |        5 | 306793 | 1612273529 |    1612273529 |         0 |                0 | 5cd5ad12e337884b4a731cb2106f77f7 |          27 |                                                                                                     |
    +--------+---------+-----------------------------------------------------------+----------------------------------+--------+-----------+----------+----------+--------+------------+---------------+-----------+------------------+----------------------------------+-------------+-----------------------------------------------------------------------------------------------------+
    
    

    There is something wrong with the md5 calculation: f24455ec205f27299f3ad1c9c51d67ff is a wrong md5, not sure where it comes from.

  16. jvillafanez commented on May 26, 2021

    @jvillafanez
    Member

    Ok, I think I've figure it out. It's basically a doble replacement that happens with the new query. We're setting the path first, then, while updating the path_hash, the path is replaced again because it matches.

    MariaDB [owncloud]> select md5('files/transferred from admin on 20210526_091542/transferred from admin on 20210526_091542/tenor.gif');
    +------------------------------------------------------------------------------------------------------------+
    | md5('files/transferred from admin on 20210526_091542/transferred from admin on 20210526_091542/tenor.gif') |
    +------------------------------------------------------------------------------------------------------------+
    | f24455ec205f27299f3ad1c9c51d67ff                                                                           |
    +------------------------------------------------------------------------------------------------------------+
    

    Inverting the order of the sets should fix the problem.

  17. jvillafanez commented on May 27, 2021

    @jvillafanez
    Member

    owncloud/core#38783 should fix the problem. I've tested the file transfer manually and I didn't notice anything weird with the patch. The md5 is the right one, and there is no duplicate in the filecache.

  18. IljaN commented on May 27, 2021

    @IljaN
    Contributor

    Confirmed: Transfer ownership tests pass now.

  19. phil-davis commented on May 27, 2021

    @phil-davis
    ContributorAuthor
  20. IljaN commented on May 27, 2021

    @IljaN
    Contributor
  21. phil-davis commented on May 27, 2021

    @phil-davis
    ContributorAuthor

    https://drone.owncloud.com/owncloud/encryption/1898/92/5
    Fetched 10.8.0 prealpha - channel: daily - build at 2021-05-26T22:00:22+00:00 - SHA: 01fd0c378e608a59c9c2b39eff302afb202e96fd

    The CI installs the current daily-master-qa tarball. That is updated every evening. We will know tomorrow morning, after the tarball is updated with the new core master code.

  22. IljaN commented on May 27, 2021

    @IljaN
    Contributor

    Thanks! I assumed PR CI always pulls latest master.

  23. phil-davis commented on May 28, 2021

    @phil-davis
    ContributorAuthor

    The transfer-ownership pipelines all passed in nightly CI: https://drone.owncloud.com/owncloud/encryption/1899

    Note: I have some other test scenarios merged to core yesterday in PR owncloud/core#38764
    That caused some fails here: https://drone.owncloud.com/owncloud/encryption/1899/212/17
    That is a separate task to adjust the skipping, nothing to do with this transfer-ownership issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions