Operations

Master

Options

A number of options are available for each configurable area within Operations. Choose the relevant area using the dropdown.

The values for each Option can be modified.

The installation typically comes pre-configured.

Concurrency

Runtime

In this section, you can set up and manage Servers, Engines, Tracks etc.

Servers

Here you maintain all the Servers in your installation setup:

You can add new servers or modify existing ones. You also delete a server, but if it is in use by Tracks, the deletion will not succeed. Add or modify will show the Server dialog:

Field Name	Description
Computer Name	This must be the name given locally on the server. If the server is an active server running one or more tracks, the Hopp Agent Windows Service will use this name to identify any track start requests destined for the server. In detail the Agent Service will get the name of the machine from the operating system, and use this name to look for the Server in the Director - so if the machine Name is not correct, the Agent will not find anything, and you will be unable to start any Tracks on the Server.
Description	The Description of the Server. The Director will show this description or the Machine Name of the Server in the UI as governed by the setting Show Servers in Director by described below (see Config).
Default track path	This is the default root folder for all the Tracks on the Server. When creating a new Track, the Director will automatically suggest best practice naming of all the folder paths for the newTrack under this root.
Network Name	The (optional) network name to use when connecting to the SQL Server instance on this Server. If blank, Machine Name will be used.
SQL Instance	An (optional) name for the SQL Instance on the Server. Leave blank for the Default Instance.
SQL Port	An (optional) port for the SQL Instance on the Server. Leave blank for the default port.
DataSource	The label DataSource will update to show the exact DataSource that is constructed from the values of Machine Name, Network Name, Sql Instance and Port.

Engines

Here you create and maintain the Engines that your Projects can use. You can add or modify Engines. If an Engine is not in use by any Project, you can also delete it. Add or Modify will show the Engine dialog:

Field Name	Description
Studio Project ID	The id assigned to this map in Studio. You can find this ID on the Settings of the root node in the Studio project Explorer tree view:
Engine name	The Engine name is displayed when you select Engines for your Projects.
Engine type	The Engine can be either a Source- or a Target Engine.
Installation path	This is the path from which the Runtime will copy the Engine when a Track starts. This can be a network location (starting with ~~) and should be reachable as a network location from the migration server(s).
Library	This is the name of the library that the Runtime will load in order to instantiate objects from the Engine. This must be the same name that is specified in Visual Studio as the assembly name for the Engine project. For the Source Engine, it must be the assembly name of the EngineCustom project.

Projects

Maintaining project engines happens here:

You can add and modify Projects. You can delete a Project if it is not running in any Tracks.

info

Deleting a Project will only remove it from the Portal management. It will not delete the Project database.

Add or Modify will show the Project dialog:

• ID: The Project ID is assigned when a new project is added.
• Name: This is the name that is displayed for the Project everywhere:.
• State: The State can be Activeor Inactive. Inactive projects are not available in the Portal. Operations are still available in the Portal. An Inactive project cannot have any Engines connected. Use the Inactive state for finished projects that only live as archives but do not have any execution tracks.
• Source- and Target Engine: Here you select the Source- and Target Engines that the Project will use
• Mode: Mode can be Export or Migration:
  • Export: Operations The project is only exporting data. In this mode, there is no Import available in the Operations for any Tracks running the Project and you cannot select a Target Engine for the Project.
  • Migration This is the full migration exporting and importing data.
• Default language: This is the language to display for the project in the Portal if a user has not selected any language in the Portals user settings.
• Project Database: Server: The Server where the Project database is located.
• Project Database: Database: The name of the Project database. Note that you must create this database yourself.
• Project Database: Connection string format: An optional connection string format to use when connecting to the database. If not specified, the global connection string maintained in the Config panel (see below) is used

Tracks

This is where you create and maintain Tracks on your Servers. You can add and modify Tracks, decide which Project uses a given Track, and finally link 2 different Tracks for Archiving (see below). You can delete a Track if it is not in use. Note that deleting a Track only removes it from the Portal management. Neither databases nor files for the track are not deleted.

Use the Filter dropdown to filter the list of tracks by:

• All
• Runtime Tracks only
• Archive Tracks only

Field	Description
ID	The Track ID is assigned when a new Track is added
Execution Server	The Server running the Track Watcher Service that will monitor for start requests for the Track. All jobs in the Track will run on this server.
Track Number	The number of the Track on the Execution Server. This must be unique for the Server. If you try to create a Track with a duplicate number, you will get an error.
Paths...	Click the Paths... button to specify the folders to use in the Track (see below). If you do not do this for a new Track, a set of default folders will automatically be created.
Archive	Check this to specify that this Track is an Archive Track and will not be executing any migration jobs. You can only change this setting if the Track is not participating in any Archiving link (see below). Archive Tracks only show up here and are hidden from the rest of the Portal.
Migration Database: Server	The Server where the Migration database is located. It is default the same as the Execution Server, but you can choose another server.
Migration Database: Database name	The name of the Migration database. Note that you must create this database yourself.
Migration Database: Connection string format	An optional connection string format to use when connecting to the Migration database. If not specified, the global connection string maintained in the Config panel (see below) is used.
Staging Database: Server	The Server where the Staging database is located. It is default the same as the Execution Server, but you can choose another server.
Staging Database: Database name	The name of the Staging database. Note that you must create this database yourself.
Staging Database: Connection string format	An optional connection string format to use when connecting to the Staging database. If not specified, the global connection string maintained in the Config panel (see below) is used.

Paths

Clicking the Paths... button brings up the Paths dialog. Here you can set the paths to the folders used by the Track. Note that these are the paths as seen from the Execution Server of the Track. It is possible to specify network share paths (starting with \\).

• Paths:
  • Runtime: The Operations will copy the Runtime as well as the Engines to be used by the Track to this folder and launch the Track governing process from here.
  • Valuesets: The Default DataServices extension delivered with Hopp will load file-based Valuesets from here
  • Source files: Source to be loaded into the Staging database must be placed in subfolders under this folder - one subfolder pr Metadata alias in the Source Map
  • Result: Operations will place the unloaded results from the migration in subfolders under this folder, one subfolder pr unload job
  • Temp: Working folder for temporary files utilized by the Track
  • Reformatted files to bulk load: During the load process, source files will be reformatted to a common format understood by Core. The reformatted files are placed in this folder
  • Bulk load format files: generate the Source Engine will generate format files to use for the SQL Server bulk load of the reformatted files into the Staging database. The Core will save the files to this folder
  • Bulk load error files: If the bulk load of source data into the Staging database produces any error files, these will be placed in this folder
  • Source duplicates: The Core contains a facility to detect duplicate lines in Source files. The result of this detection is placed in this folder
  • Archiving: This folder is not active yet but will be playing a role in the coming release of automatic archiving of the Runtime track
  • Extensions: This folder will be provided for any extensions running as jobs under the Core Runtime
• Root folder: Clicking the Set Defaults button will automatically create a suggested best-practice hierarchy of paths under the specified Root folder
• Set Defaults: Click the button automatically create a suggested best-practice hierarchy of paths under the specified Root folder

Usage

Click the Usage button to assign a Track to be used by a given Project - or to release the Track for use by other Projects.

• Vacant: The Track is vacant and not in use by any Project
• In Use: The Track is in use by a Project
• Project: Choose the Project using the Track
• Usage: Give this usage of the Track a label. This label will show up in the Portal
• Publish: The Track will not show up anywhere in the Portal if Publish is unchecked. Use this setting to keep an new track hidden while you are setting everything up
• Tracker Expiry: If set, the Usage will not be visible in the Portal past this date

Archiving

Linking Tracks for Archiving will enable the automatic archiving:

You only can link a Runtime Track to an Archive Track being used by the same Project. An automatic Archive will:

• Backup the Archive databases and files to the Archiving folder of the Archive Track.
• Backup the Runtime databases and files and move them to the Archiving folder of the Archive Track
• Restore and replace the Archive databases with the Runtime backup
• Delete the backup of the Runtime databases

Secrets

Hopp loads Metadata and dynamic Valuesets from DataService Provider extensions. A Provider to read Metadata and Valuesets from Excel is baked into Hopp from the beginning. Hopp can provide additional DataService Providers to get Metadata and Valuesets from for instance SQL Server, Oracle or DB2. Finally, bespoke DataService Providers can be developed to get Metadata and Valuesets from other sources.

The SQL Server, Oracle, and DB2 DataServices from Hopp all require a connection string as parameter. In some cases, it may be necessary to add sensitive information - a password - to a connection string. Other DataServices may also require sensitive information in parameters.

This poses a problem because the DataServices parameters are visible to all users in Studio and - if provided via a Runtime Constant - also visible to all users in Director and even Business Users in the Portal.

Administration of Secrets

To get around this, Hopp supports Secrets. A Secret is a named value, and the value of the Secret is maintained in the Portal Configuration:

Secrets are stored in the Master database with an encrypted secret value.

Config

In the Config section, you can edit a few global configuration settings.

tip

Remember to click the Save button if you make any changes.

• Default connection string format: This is the format the Core Runtime will use for the connection strings. In a given context, it will replace the placeholder string @server with the actual Data source of the given SQL Server instance and the placeholder string @database with the database name. Normally this global format is all you need to specify, but as you have seen above, it is possible to override this format in specific instances. Notice that the connection string setting MultipleActiveResultsets is set to true. This setting is vital for the Core Runtime.
• Web Service URL for Translation Valuesets: This is the URL all Tracks will use when they need to load the Translation Valuesets from the Portal to the Director Runtime. The server part of the URL will normally be the same as the URL type in the browser to get hold of the Portal web application
• Director client pulse interval: This interval determines how often all connected Operations will poll the master server for configuration changes. It is preset to 2500 milliseconds which is normally fine. You only need to touch this setting if there are extraordinarily any clients connected and the polling is imposing a heavy load on the master server. Normally this will never be the case.
• Show Servers in Director by: Determine whether Servers will be shown in Operations by their Machine Name or by their Description. This will change the way Tracks are shown in the Director menu as well as in the Title bar of any Track window
• Additional configuration settings: Here you can add additional settings that will be automatically added to the Track config file when a track is started. You can add either connection strings (will be added to the connectionStrings section) or application settings (will be added to the appSettings section). This is useful if you develop extensions to the Core Runtime that need their own settings in the config file, for instance, a connection string to your own Master database.

Upload

This screen allows version release files to be uploaded and managed.