Starting a custom Drupal 8 Migration

There will be the occasional project coming in from Wordpress, Joomla!, static HTML, classic ASP.net sites or other sources. In other cases, you may not want to do the assisted core migration, but craft your own migration procedure. 

Fortunately, there are tools being worked on to facilitate requirements like this. Migrate Plus and Migrate Tools are in progress and introduce functionality to Drupal 8 migrations that will feel familiar to developers who worked extensively with the Migrate Framework from before.

Here is an example of a manual migration of roles, users, and user fields from Drupal 7 to Drupal 8. To do this, you will need the aforementioned Migrate Plus and Migrate Tools modules, a defined source in your Drupal 8 settings.php configuration, and define a migration group. You will need to plug in your database connection key in the group yaml definition, which points the migration at the database connection to use as the source of migrations. 

All migrations within the same group will use the same data source thanks to the shared_configuration values in your migration group. In more advanced scenarios, you may have multiple data sources defined.

Let's take a look at a basic example. In this case, I wanted to bring in user roles and all of our user accounts, the role assignments and couple of fields attached to user accounts that we use on velir.com.

With the core migration, a few things were happening. The 'administrator' role was being recreated as 'administrator1', and the custom field values were not coming through. Obviously not ideal, so I needed to splinter off and do my own migration.

To start with, I created a module, a migration group. I then copied in the core user role and user migration template to my module and adjusted them as follows:

In migrate_plus.migrate_group.velir.yml:

# The machine name of the group, by which it is referenced in individual
# migrations.
id: mymigration

# A human-friendly label for the group.
label: Custom Migration

# More information about the group.
description: "My custom migrations"

# Short description of the type of source, e.g. "Drupal 6" or "WordPress".
source_type: "Drupal 7"

# Here we add any default configuration settings to be shared among all
# migrations in the group. For this example, the source tables are in the
# Drupal (default) database, but usually if your source data is in a
# database it will be external.
shared_configuration:
  source:
    key: d7

In migrate_plus.migration.user_role.yml:

id: user_role
label: User roles
migration_group: mymigration
migration_tags:
  - Drupal 7
source:
  plugin: d7_user_role
process:
  id:
    -
      plugin: machine_name
      source: name
    -
      plugin: substr
      entity_type: user_role
      field: id
      length: 32
    -
      plugin: user_update_8002
  label: name
  permissions:
    -
      plugin: static_map
      source: permissions
      bypass: true
      map:
        'use PHP for block visibility': 'use PHP for settings'
        'administer site-wide contact form': 'administer contact forms'
        'post comments without approval': 'skip comment approval'
        'edit own blog entries' : 'edit own blog content'
        'edit any blog entry' : 'edit any blog content'
        'delete own blog entries' : 'delete own blog content'
        'delete any blog entry' : 'delete any blog content'
        'create forum topics' : 'create forum content'
        'delete any forum topic' : 'delete any forum content'
        'delete own forum topics' : 'delete own forum content'
        'edit any forum topic' : 'edit any forum content'
        'edit own forum topics' : 'edit own forum content'
    - plugin: flatten
  weight: weight
destination:
  plugin: entity:user_role
migration_dependencies: {}

In migrate_plus.migration.user.yml:

id: user
label: Users
migration_group: mymigration
source:
  plugin: user
  keys:
    - uid
destination:
  plugin: entity:user
process:
  uid: uid
  pass: pass
  name: name
  mail: mail
  init: mail
  status: status
  created: created
  changed: created
  access: access
  login: login
  roles:
    plugin: migration
    migration: user_role
    source: roles
  field_full_name: field_full_name
  field_job_title: field_job_title
  field_weight: field_weight
migration_dependencies:
  required:
    - user_role

Bit of a long way around, but fixed my immediate issue. Since the substr process plugin is now in core, I replaced the core default of 'dedupe_entity' with 'substr'. This prevents my administrator role being recreated as administrator1, and all users were assigned the appropriate roles.

Secondly, the basic fields were mapped to their Drupal 8 fields. An extra step to this process was also creating the fields before doing the migration, whereas the cores assisted migration will create them for you. You can mimic this pattern in your module (copy the requisite migration templates from the field module and define them in your module), and set that as a dependency before migrating your entities.