as stated by chx at #1209532: Count node views via AJAX in the statistics module, despite what the doxygen says db_merge does not allow multivalue inserts.

at http://api.drupal.org/api/drupal/includes!database!query.inc/class/Merge...
INSERT (column1 [, column2 ...]) VALUES (value1 [, value2 ...

but MergeQuery::execute runs
$this->connection->insert($this->table)->fields($this->insertFields)->execute()

which is not using $insertValues anywhere.

We need to add an insertValues method and change execute to work with it and add unit tests.

Comments

lucascaro’s picture

Assigned: Unassigned » lucascaro
Status: Active » Needs review
StatusFileSize
new777 bytes

Ok, here is my first attempt to move this issue forward.
Since the problem is that there is no way to add values to a merge query, I've added a simple test case that checks for the existence of the methods value and insertValues.
Please let me know if there is a better way to start. I'm planning on adding those two methods and then create some tests for them.

Is this test I'm posting now really needed or is there another way to do this?

Status: Needs review » Needs work

The last submitted patch, database_multiple_insert_test-1499738-1.patch, failed testing.

sun’s picture

Issue summary: View changes

Hrm. Conceptually, this doesn't make sense to me. The UPDATE query would update the row specified by key(), but the INSERT query would potentially insert more?

Aren't we massively diverging from built-in database driver support for MERGE queries with this? (which we're not leveraging, but those should still lead the conceptual design)

danblack’s picture

Assigned: lucascaro » danblack
Issue tags: +API addition

> diverging from built-in database driver support for MERGE

Which driver has built in driver support for merge?

I see db_merge with multiple values as a nice convince function rather than convoluted module implementations (#512962-273: Optimize menu_router_build() / _menu_router_save()).

Where unique key(s) are specified to the merge can become.

db_transaction();
$pks = SELECT pk FROM tbl WHERE ...... FOR UPDATE
for ($values as $v) {
..$pk = extract_unique($v)
..UPDATE tbl ($updateFields) ..SET .... WHERE pk = $pk 
}
INSERT INTO tbl ($insertFields) VALUES .....

(or just REPLACE / ON DUPLICATE KEY UPDATE[#965646] in MYSQL).

I may have missed come concept here. Perhaps implementing #1800286: Update, rather than Delete and insert - for significant database performance improvement). I may come across something new. Like it says here #310085: Merge queries using db_merge some things may not make sense.

Doing multivalue changes will also mean the return code can't be a STATUS_INSERT or STATUS_UPDATE. I guess since multiple values are here this is really an API addition and the STATUS code could be changed to STATUS_BOTH_INSERT_UPDATE if really needed.

danblack’s picture

danblack’s picture

Lets look at which multivalue db_merge statements make sense and don't:

Works

update fields/values == insert fields/values where one of the columns is a key

This is the case for bulk updates.

$values = array(
  array('col1' => 'green', 'col2' => 'plant'),
  array('col1' => 'red', 'col2' => 'sun'), 
);

db_merge('table')->key('col1')->fields($values);

update is expression

db_merge('table')->key('col')->insertFields($values)->expression('col3 = col3 +1');

merge has constant condition

Update the values where for active rows and otherwise insert them.

      db_merge('test_people')
        ->key('col1')
        ->condition('active',TRUE,'=')
        ->fields($values)
        ->execute();

merge has multiple inserts (and key isn't unique)

inserts two rows when found otherwise updates those rows (assumes col1 isn't unique).

      db_merge('test_people')
        ->key('col1','red')
        ->condition('col2','sun','=')
        ->insertFields(array('col2' => 'flower'), array('col2'=>'car', 'col3' => 'fast')
        ->updateFields('col3' => 'super nova')
        ->execute();

Doesn't work

multiple insert values where key is unique

    db_merge('test_people')
      ->key('job', 'Speaker')
      ->insertFields(array(
        array('age' => 31, 'name' => 'Tiffany'),
        array('age' => 19, 'name' => 'Bob'),
      ))
      ->updateFields(array(
        'name' => 'Joe',
      ))
      ->execute();

merge has multiple inserts and key

Can't insert two rows as unique on col1=red would be violated on second insert.

      db_merge('test_people')
        ->key('col1','red')
        ->condition('col2','sun','=')
        ->insertFields(array('col2' => 'flower'), array('col2'=>'car', 'col3' => 'fast')
        ->updateFields('col3' => 'super nova')
        ->execute();

(most probably incomplete). Is there enough valid cases here to throw InvalidMergeQueryException for the invalid cases?

danblack’s picture

Assigned: danblack » Unassigned

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

joelpittet’s picture

This could use a subsystem maintainer's response on this as it's getting a bit stale

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

amateescu’s picture

Status: Needs work » Closed (outdated)

I think the new UPSERT query in core and the proposed patch from #2547493: Add support for unique / primary key constraints composed of multiple fields for Upsert queries makes this issue obsolete.