1. Announcements

1.1 SVN Read-only

The FESA SVN repository will be set to read only in the last week of october. All local changes should be committed from workspace to SVN latest on Friday 23.10.2020.

1.2 FESA Mass Migration

Mass migration of all FESA projects in https://www-acc.gsi.de/viewvc/view/fesa/device/ that are not yet migrated will start on Monday 26.10.2020.

• There are some projects that cannot migrated with the script. I will contact the respective developers in the coming weeks to address those issues.
• Please inform me when you can identify Projects that are no longer needed. For example DUMMY, DUMMY2 and DUMMY3 are FESA class projects of mine that should not be in SVN in the first place.
• If I find projects that already exist in Git, I will assume that they are already migrated and not take any further action.
  • If you do not want to migrate your own projects you can ignore this note.
  • If you do want to migrate your own projects (see below) it is up to you to properly migrate the project or inform me in case of problems.

2. .gitignore

The .gitignore of FESA Git repositories should have the following entries:

• bin
• build
• precompiled
• generated
• Makefile
• Makefile.dep
• .settings

Some SVN projects contain files that actually should have been svn-ignored. We want to use the migration to clean ignored files from the repositories.

If you use the script described below you do not have to do anything extra. The script not only adds the .gitignore but also removes ignored files if present. In the rare case that this will remove files that you actually need, the last git commit can tell you what files have been removed and in the previous commit they are still present.

If you migrate FESA projects by any other means, we kindly ask you to clean ignored files.

3. FESA Git-Migration scripts

The FESA migration scripts can be found here: https://git.acc.gsi.de/thaberm/fesa_migration.git. The scripts are based on https://git.acc.gsi.de/schaller/GitMigrationScript. The original script had to me modified to account for non standard SVN layouts of FESA projects. The modified git_migration.sh can be used also to migrate non FESA projects. For FESA projects it is not recommended to use git_migration.sh directly. There are helper scripts that are easier to use (but less flexible). Only use the git_migration.sh directly when you know what you are doing.

3.1 Migratation scripts: General

Detailed information for the original git_migration.sh script and migration to Git in general can be found here: https://www-acc.gsi.de/wiki/Applications/AppHowToGitMigrationOfSvnProject. Note that the documentation is written for AP and for FESA projects some points do not apply:

• 1. Choosing or creating the corresponding organization in Git
  • you do not have to choose or create a Git organization. The Git organisations for FESA projects are listed below.
• 3.2 Get the migration script
  • get the modified script from https://git.acc.gsi.de/thaberm/fesa_migration.git
• 3.3 Executing the migration script
  • most FESA projects cannot be migrated with the original git_migration.sh
  • instead of using the modified git_migration.sh it is recommended to use either fm_migrate.sh or fm_mass_migrate.sh (see below for details).
• 4.1 Oooops, we have SVN updates after migrating the project to Git
  • we want to avoid this situation (see next point)
  • you cannot migrate later changes on the SVN history to Git for projects migrated with the FESA migration scripts!
    (we add one commit to Git but not to SVN, but the update script only works one way. Also fm_mass_migrate.sh (see below) removes the local clone from disc after successful migration)
  • the easiest way to synchronize would be to delete the Git repository and migrate it again.
  • once SVN is set to read only this cannot happen
• 4.2 Prevent changes to SVN
  • for FESA projects migrated before SVN is set to read-only you should follow this suggestion.

3.1.1 SVN Layout

Most FESA projects have one of the following svn layouts:

svn_layout	./
svn_layout_std	trunk/ tags/ branches/
svn_layout_tag	trunk/ tag/ branches/
svn_layout_tag2	trunk/ tag/ tags/ branches/
svn_layout_none	src/ and no trunk/tag/tags/branches)

Projects that have a different structure (svn_layout_other) cannot be migrated with the script.

3.1.2 Category

The scripts use one argument to select the "category" of FESA projects:

category	svn_path	git organization
class	class	fesa-classes
deploy	deploy-unit	fesa-deploy-units
driver	driver	fesa-drivers

3.1.3 misc

• The actual migration script (git_migration.sh) will create a clone of the repository to be migrated in the current directory. After sucesfully migrating the repository you can either remove that clone from disc or move it to your actual workspace. Note that the script to migrate several projects (fm_mass_migrate.sh) will remove the local clone after a sucesful migration.
• To avoid retyping your login you can create a gitea token: https://git.acc.gsi.de/user/settings/applications. The scripts excepts that to be in a file called token in a directory called target.
• git-svn requires a file containing a list of all usernames, their full name and email adresses. We use the file fm_users.txt. If you find that a user is missing from that file (you will get an error during migration) please let me know so we can update that file.

3.2 fm_svn_info.sh : Listing FESA SVN Projects

Optional Argument: user

Display information for all FESA projects:

> fm_svn_info.sh
class    A                              svn_layout_std     matthies   r16776-2018-03-07-08:43:03
class    AA                             svn_layout_std     matthies   r16779-2018-03-07-08:59:51
class    ABC                            svn_layout_std     matthies   r16935-2018-03-19-12:02:30
[...]

The columns are: category, name of the project, the svn layout, author of last commit, revision and timestamp of last commit. Only the first two columns are required as input for fm_mass_migrate.sh. The last three are only for your information:

• Layout: The layout is determined according to the above table. Projects with svn_layout_other cannot be migrated with the script!
• Author: We only consider the last commit to determine the author of the project.
• Revision: The revision number will also appear in the git commits and can be used to verify that Git and SVN are in sync.

Display projects for one user only:

> fm_svn_info.sh thaberm
class    B1EventSourceTest              svn_layout_std     thaberm    r20698-2019-11-06-16:07:34
class    DUMMY                          svn_layout_std     thaberm    r21556-2020-09-08-16:47:03
class    DUMMY2                         svn_layout_std     thaberm    r21562-2020-09-08-16:49:33
class    DUMMY3                         svn_layout_std     thaberm    r21565-2020-09-08-16:50:10
[...]

Note that if the last commit was made by someone else the project will not be listed.

3.3 fm_migrate.sh : Migrating a single project

Arguments: category project_name

fm_migrate.sh migrates a single fesa project. Example:

> fm_migrate.sh class MASP
[ INFO  ] fm_migrate
[ INFO  ]               svn_category   class
[ INFO  ]               svn_proj_name  MASP
[ INFO  ]               svn_path       /device/class
[ INFO  ]               url            https://www-acc.gsi.de/svn/fesa/device/class/MASP
[ INFO  ]               git_orga       fesa-classes
[ INFO  ] check if organization fesa-classes has a repository named MASP
[  OK   ] MASP does not exist in fesa-classes
[ INFO  ] determine svn layout
[  OK   ] MASP has layout: svn_layout_std
[ INFO  ] call the actual migration script:/common/home/bel/thaberm/lnx/git/fesa_migration/git_migration.sh --svnAuthorsFile /common/home/bel/thaberm/lnx/git/fesa_migration/fm_users.txt --project MASP --svnGroup fesa --svnPath /device/class --svnLayout=svn_layout_std --organization fesa-classes --add-gitignore

[... output of git_migration.sh ...]

The script

• checks if a repository of the same name already exists in the respective Git organisation
• determines the SVN layout of the project
• calls the actual script: git_migration.sh

After running the script all history of the project will be available in Git. You can verify that for example by looking at the revision number of the last SVN commit

> svn info  https://www-acc.gsi.de/svn/fesa/device/class/MASP
Path: MASP
URL: https://www-acc.gsi.de/svn/fesa/device/class/MASP
Repository Root: https://www-acc.gsi.de/svn/fesa
Repository UUID: 243f544d-f450-4db5-8dff-af105d85f54e
Revision: 21600
Node Kind: directory
Last Changed Author: thaberm
Last Changed Rev: 21569
Last Changed Date: 2020-09-08 17:19:50 +0200 (Tue, 08 Sep 2020)

The last SVN revision is 21569 (21600 is the revision for the complete FESA repository). From the local clone we can see the last two commits in the newly created Git repository:

> cd MASP
> git log --all -n 2
commit 15c94d9b101beb07fef558c23c6c3cff1558e6d1
Author: thaberm <thaberm@asl744.acc.gsi.de>
Date:   Wed Sep 23 14:47:06 2020 +0200

    GIT Migration from SVN: add .gitignore and removed ignored

commit e2f526c2a6be2381730c10907ebc04241ffdbf81
Author: Toibas Habermann <t.habermann@gsi.de>
Date:   Tue Sep 8 15:19:50 2020 +0000

    AutomaticTagCreatedDuringReleaseOfOperationalVersionV1_7_3
    
    git-svn-id: https://www-acc.gsi.de/svn/fesa/device/class/MASP/tags/V1_7_3@21569 243f544d-f450-4db5-8dff-af105d85f54e

The last commit is from git_migration.sh adding the .gitignore. This commit is not present in SVN. The previous commit corresponds to the last SVN commit and the SVN revision 21569 is part of the git-svn-id.

3.4 fm_mass_migrate.sh : Migrating many projects

Argument: svn_list (file with list of projects).

fm_mass_migrate.sh takes as input a file with a list of FESA projects. The list must contain the category and the name of the projects. Output from fm_svn_info.sh can be used, the other columns are then ignored. Example:

> fm_svn_info.sh mmustermann > my_projects.txt
> fm_mass_migrate.sh my_projects.txt
[ INFO  ] fm_mass_migrate. Migrating all projects listed in my_projects.txt
[ INFO  ] List of sucessfully migrated projects will be written to report2020-09-23-14-52-54.txt
[...]
[...]
[ INFO  ] =======================================================================================
[ INFO  ] fm_mass_migrate finished. List of sucesfully migrated project is written to report2020-09-23-14-52-54.txt

For each listed project the script will perform the same actions as fm_migrate.sh. In addition:

• when there was no error
  • the local clone of the newly created git repository is removed from disc.
  • the category and name of the project is written to a report file. This file can be used to resume fm_mass_migrate.sh in case of errors. In the above example the file is called report2020-09-23-14-52-54.txt

3.5 fm_rep2resume.sh : Resume in case of Errors

Arguments: svn_file report_file

The report file created by fm_mass_migrate.sh lists all sucessfully migrated projects. You can use the orignal intput to fm_mass_migrate.sh and the report file to resume migration:

> fm_mass_migrate.sh my_projects.txt
... some projects have errors
> fm_rep2resume.sh my_projects.svn report2020-09-23-14-52-54.txt > next_trial.txt
... fix problems and try again
> fm_mass_migrate.sh next_trial.txt

3.6 git_migration.sh : "expert mode"

The original script had to be modified to account for non standard svn layout of FESA Projects. More modifications include:

• cosmetics: no more colors in ouput
• svn-layouts: support for most common layouts of FESA-Projects (and no rootTTB layout)
• gitignore: when a gitignore is added to the git repo the script adds a commit to remove ignored files from the repository, but that last commit is not synced with svn (!)

Arguments for fesa-classes are:

• --project the name of the project
• --svnGroup fesa
• --svnPath /device/class or /device/driver or /device/deploy-unit
• --organization fesa-classes / fesa-deploy-units / fesa-drivers
• --svnLayout=...see above..., for example --svnLayout=svn_layout_std (no spaces!)
• --add-gitignore
• --svnAuthorsFile ${DIR}/fm_users.txt

Beware

• You can use this script directly, but there are no safety nets included. Using the wrong argument for svnLayout may cause loss of tags / branches or parts of the history.
• git does not store empty directories. When there are empty directories in svn they will be ignored during migration.

-- TobiasHabermann - 16 Sep 2020