Url https://blog.werk21.de/en en Change source id mapping of a Drupal 8 migration https://blog.werk21.de/en/2017/12/13/change-source-id-mapping-drupal-8-migration <span>Change source id mapping of a Drupal 8 migration</span> <span><span lang="" about="/en/user/8" typeof="schema:Person" property="schema:name" datatype="">patrick</span></span> <span>Wed, 12/13/2017 - 06:57</span> <div class="field field--name-body field--type-text-with-summary field--label-hidden field--item"><p>Drupal Core's Migrate API maintains an internal table mapping unique source ids to destination ids. This is usually done via a database table that holds the field values that together uniquely identify a source row and the matching destination id of the entity in Drupal. The field names and types that uniquely identify a source row are provided by the source plugin of a migration. In most cases, you just have some kind of an id column, but migrate supports any number of fields for the primary key of the source row.</p> <p>Recently I was dealing with a migration that was using the <span class="geshifilter"><code class="php geshifilter-php">Url</code></span> source plugin provided by <a href="https://drupal.org/project/migrate_plus">Migrate Plus</a> to import files from an <span class="geshifilter"><code class="php geshifilter-php">XML</code></span> source. The source plugins provided by Migrate Plus allow you to specify the ids returned by <a href="https://api.drupal.org/api/drupal/core%21modules%21migrate%21src%21Plugin%21MigrateSourceInterface.php/function/MigrateSourceInterface%3A%3AgetIds/8.4.x">MigrateSourceInterface::getIds()</a> in the <span class="geshifilter"><code class="php geshifilter-php">YML</code></span> file of your migrations. Originally I used a simple id map like this:</p> <div class="geshifilter"><pre class="php geshifilter-php" style="font-family:monospace;">source<span style="color: #339933;">:</span> ids<span style="color: #339933;">:</span> id<span style="color: #339933;">:</span> type<span style="color: #339933;">:</span> integer</pre></div> <p>Unfortunately, I later noticed that I needed to add the file name as a second key to the id map, so I changed the YML file to something like this:</p> <div class="geshifilter"><pre class="php geshifilter-php" style="font-family:monospace;"> ids<span style="color: #339933;">:</span> id<span style="color: #339933;">:</span> type<span style="color: #339933;">:</span> integer file_name<span style="color: #339933;">:</span> type<span style="color: #339933;">:</span> string</pre></div> <p>So far so good. When I wanted to update the migration, I was greeted by this error message:</p> <blockquote> <p>Migration failed with source plugin exception: SQLSTATE[42S22]: Column not found: 1054 Unknown column "sourceid2" in<br /> "field list": INSERT INTO {migrate_map_example_file} (source_ids_hash, sourceid1, sourceid2, source_row_status,<br /> rollback_action, hash, last_imported) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2,<br /> :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6); Array<br /> (<br />     [:db_insert_placeholder_0] =&gt; a8d0cbb8ca5f48c73993244fd275342d84b92a2eb33300b0852ef560f70aa55d<br />     [:db_insert_placeholder_1] =&gt; 10817<br />     [:db_insert_placeholder_2] =&gt;<br />     [:db_insert_placeholder_3] =&gt; 2<br />     [:db_insert_placeholder_4] =&gt; 0<br />     [:db_insert_placeholder_5] =&gt;<br />     [:db_insert_placeholder_6] =&gt; 1513082182<br /> )  </p> </blockquote> <p>Solving this is actually way easier than I thought. Just drop the migrate mapping table for your migration from the SQL database:</p> <div class="geshifilter"><pre class="php geshifilter-php" style="font-family:monospace;">DROP TABLE migrate_map_example_file<span style="color: #339933;">;</span></pre></div> <p>If you are less adventurous, you might just want to rename the table initially:</p> <div class="geshifilter"><pre class="php geshifilter-php" style="font-family:monospace;"><a href="http://www.php.net/rename"><span style="color: #990000;">RENAME</span></a> TABLE migrate_map_example_file TO migrate_map_example_file_backup<span style="color: #339933;">;</span></pre></div> <p>As soon as you rerun your migration, migrate will recreate the table for you, this time using the new id mapping. Voilá.</p> </div> <div class="field field--name-field-systems field--type-entity-reference field--label-inline"> <div class="field--label">Systems</div> <a href="/de/systems/drupal-8" class="label label-default" hreflang="de">Drupal 8</a> </div> <div class="field field--name-field-tags field--type-entity-reference field--label-inline"> <div class="field--label">Tags</div> <a href="/de/tags/migrate" class="label label-default" hreflang="de">migrate</a> <a href="/en/tags/migrateplus" class="label label-default" hreflang="en">migrate_plus</a> <a href="/en/tags/source" class="label label-default" hreflang="en">source</a> <a href="/en/tags/url" class="label label-default" hreflang="en">Url</a> <a href="/en/tags/migratesourceinterface" class="label label-default" hreflang="en">MigrateSourceInterface</a> <a href="/en/tags/getids" class="label label-default" hreflang="en">getIds</a> </div> Wed, 13 Dec 2017 05:57:34 +0000 patrick 128 at https://blog.werk21.de Change list of valid URI schemes in Drupal 8 https://blog.werk21.de/en/2017/06/06/change-list-valid-uri-schemes-drupal-8 <span>Change list of valid URI schemes in Drupal 8</span> <span><span lang="" about="/en/user/8" typeof="schema:Person" property="schema:name" datatype="">patrick</span></span> <span>Tue, 06/06/2017 - 13:12</span> <div class="field field--name-body field--type-text-with-summary field--label-hidden field--item"><p>If you need to support <em>obscure</em> <a href="https://en.wikipedia.org/wiki/Uniform_Resource_Identifier">URI schemes</a> in your Drupal 8 instance and you're getting error messages about invalid URI schemes, this post is for you.</p> <p>Suppose a customer asks you to add a <em>WhatsApp</em> Share Link somewhere. It might look something like this:</p> <div class="geshifilter"><pre class="php geshifilter-php" style="font-family:monospace;"><span style="color: #000000; font-weight: bold;">use</span> \Drupal\Core\Url<span style="color: #339933;">;</span> &nbsp; Url<span style="color: #339933;">:</span>fromUri<span style="color: #009900;">&#40;</span><span style="color: #0000ff;">'whatsapp://send'</span><span style="color: #339933;">,</span> <span style="color: #009900;">&#91;</span> <span style="color: #0000ff;">'query'</span> <span style="color: #339933;">=&gt;</span> <span style="color: #009900;">&#91;</span> <span style="color: #0000ff;">'text'</span> <span style="color: #339933;">=&gt;</span> <span style="color: #0000ff;">'Some text'</span><span style="color: #339933;">,</span> <span style="color: #009900;">&#93;</span><span style="color: #339933;">,</span> <span style="color: #009900;">&#93;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">-&gt;</span><span style="color: #004000;">toString</span><span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span></pre></div> <p>This should work, since the URI is perfectly valid per the specification. But instead of the desired link, we'll get this error message, that results in a white page:</p> <blockquote> <em>InvalidArgumentException</em>: The URI 'whatsapp://send' is invalid. You must use a valid URI scheme. Use base: for a path, e.g., to a Drupal file that needs the base path. Do not use this for internal paths controlled by Drupal. in <em>Drupal\Core\Utility\UnroutedUrlAssembler-&gt;assemble()</em> (line <em>64</em> of <em>core/lib/Drupal/Core/Utility/UnroutedUrlAssembler.php</em>). </blockquote> <p>What happened here? Drupal 8 actually maintains a list of <em>valid</em> URI schemes, that are allowed to be used during URL generation. If a scheme is not in this list of protocols, it will result in this error message. <span class="geshifilter"><code class="php geshifilter-php">whatsapp</code></span> is not included in this list by default, so Drupal considers the URI to be invalid.</p> <p>This is actually a nice security feature, but what if we just need to support schemes other than the default? It's easy and quickly done, as long as you know where to look. The allowed protocols are specified as a parameter to the services container. The parameter in question is called <span class="geshifilter"><code class="php geshifilter-php">filter_protocols</code></span>. The default parameters for the services container are set in a file called <span class="geshifilter"><code class="php geshifilter-php"><span style="color: #b1b100;">default</span><span style="color: #339933;">.</span>services<span style="color: #339933;">.</span>yml</code></span> in the <span class="geshifilter"><code class="php geshifilter-php">sites<span style="color: #339933;">/</span><span style="color: #b1b100;">default</span></code></span> directory of your Drupal installation. You could also use a <span class="geshifilter"><code class="php geshifilter-php">development<span style="color: #339933;">.</span>services<span style="color: #339933;">.</span>yml</code></span> file just for development and more. If you want to change any of the settings, copy the <span class="geshifilter"><code class="php geshifilter-php"><span style="color: #b1b100;">default</span><span style="color: #339933;">.</span>services<span style="color: #339933;">.</span>yml</code></span> file to <span class="geshifilter"><code class="php geshifilter-php">services<span style="color: #339933;">.</span>yml</code></span> and change the settings there as needed. To support e.g. the <span class="geshifilter"><code class="php geshifilter-php">whatsapp</code></span> protocol, we would need to add that to the list defined by the <span class="geshifilter"><code class="php geshifilter-php">filtered_protocols</code></span> parameter. Now rebuild your container (e.g. by calling <span class="geshifilter"><code class="php geshifilter-php">drush cr</code></span> on the command line) and the error message should be gone.</p> <div class="geshifilter"><pre class="php geshifilter-php" style="font-family:monospace;"><span style="color: #666666; font-style: italic;"># Allowed protocols for URL generation. </span>parameters<span style="color: #339933;">:</span> filter_protocols<span style="color: #339933;">:</span> <span style="color: #339933;">-</span> http <span style="color: #339933;">-</span> https <span style="color: #339933;">-</span> ftp <span style="color: #339933;">-</span> news <span style="color: #339933;">-</span> nntp <span style="color: #339933;">-</span> tel <span style="color: #339933;">-</span> telnet <span style="color: #339933;">-</span> mailto <span style="color: #339933;">-</span> irc <span style="color: #339933;">-</span> ssh <span style="color: #339933;">-</span> sftp <span style="color: #339933;">-</span> webcal <span style="color: #339933;">-</span> rtsp <span style="color: #339933;">-</span> whatsapp</pre></div> <p>Of course you could also remove any schemes you don't need, e.g. <span class="geshifilter"><code class="php geshifilter-php">telnet</code></span> for increased security ;).</p></div> <div class="field field--name-field-systems field--type-entity-reference field--label-inline"> <div class="field--label">Systems</div> <a href="/de/systems/drupal-8" class="label label-default" hreflang="de">Drupal 8</a> </div> <div class="field field--name-field-tags field--type-entity-reference field--label-inline"> <div class="field--label">Tags</div> <a href="/en/tags/servicesyml" class="label label-default" hreflang="en">services.yml</a> <a href="/en/tags/url" class="label label-default" hreflang="en">Url</a> </div> Tue, 06 Jun 2017 11:12:50 +0000 patrick 121 at https://blog.werk21.de