Move a directory to a different git repository while keeping its history

When you simply move a directory into another git repository the history of the files is gone. With the following steps you can move and keep the history of a single directory.

Migration Steps

The basic procedure is from this article https://gbayer.com/development/moving-files-from-one-git-repository-to-another-preserving-history/ extended by moving the files back to their package path in history and optionalo rebasing instead of merging the history. This makes it look better and easier if you want to rebase ongoing changes on top of it later.

Prepare the script under move_in_history seen below as "Move in history script" in your filesystem.

git clone <git repository A url>
cd <git repository A directory>
git remote rm origin
git filter-branch --subdirectory-filter <directory 1> -- --all
~/move_in_history
git add .
git commit

Merge files into the new repository

Make a copy of repository B if you don’t have one already. On line 3, you’ll create a remote connection to repository A as a branch in repository B. Then simply pull from this branch (containing only the directory you want to move) into repository B. The pull copies both files and history. Note: You can use a merge instead of a pull, but pull worked better for me. Finally, you probably want to clean up a bit by removing the remote connection to repository A. Commit and you’re all set.

git clone <git repository B url>
cd <git repository B directory>
git remote add repoA <git repository A directory>
git pull --no-rebase repoA master

# use the following rebase if your repository B is a newly created repository or else stay
# wit the created merge if it already existed, to show the merged histories clearly
# git rebase origin/master

git remote rm repoA

--allow-unrelated-histories might be necessary while doing the git pull

Move in history script

cat ~/move_in_history

#!/bin/bash
# Replace DIRECTORY1 with "directory 1" EXACTLY (keep & before and - after, have a SLAH AT THE END of the dir)
# (Do not confuse the slash with the regex separator, this is "-" is this case.
git filter-branch -f --index-filter \
                   'git ls-files -s | sed "s-\t\"*-&DIRECTORY1/-" |
                           GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
                                   git update-index --index-info &&
                    mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD

(From git help filter-branch)

Example Scenario

In this hypothetical example we want to move de.gsi.fcc.screenshot.app.util.MathUtil from screenshot-app to de.gsi.cs.co.ap.common.utils in common-utils.

Source

Destination

Procedure

$ mkdir -p ~/tmp/repomove
$ cd ~/tmp/repomove

$ git clone git@git.acc.gsi.de:fcc-applications/screenshot-app.git
Cloning into 'screenshot-app'...
remote: Counting objects: 1262, done.
remote: Compressing objects: 100% (413/413), done.
remote: Total 1262 (delta 456), reused 1221 (delta 443)
Receiving objects: 100% (1262/1262), 751.58 KiB | 0 bytes/s, done.
Resolving deltas: 100% (456/456), done.

$ git clone git@git.acc.gsi.de:fcc-commons/common-utils.git
Cloning into 'common-utils'...
remote: Counting objects: 1074, done.
remote: Compressing objects: 100% (366/366), done.
remote: Total 1074 (delta 286), reused 1017 (delta 277)
Receiving objects: 100% (1074/1074), 124.86 KiB | 0 bytes/s, done.
Resolving deltas: 100% (286/286), done.

$ cd screenshot-app 
# for safety reasons
$ git remote rm origin 
$ ls src/main/java/de/gsi/fcc/screenshot/app/util                                              
MathUtil.java
$ git filter-branch --subdirectory-filter src/main/java/de/gsi/fcc/screenshot/app/util -- --all
Rewrite 71907bb65156911dc47b2402f19258dafe764603 (3/3)
Ref 'refs/heads/master' was rewritten
WARNING: Ref 'refs/tags/app-screenshot-10.0.0' is unchanged
[...]
WARNING: Ref 'refs/tags/app-screenshot-13.0.2' is unchanged

$  ls
MathUtil.java

$  vim ~/move_in_history
$  ~/move_in_history    
Rewrite d5e8f0b55b9a329933bc700a99cb9efa61923092 (3/3)
Ref 'refs/heads/master' was rewritten

$  ls src/main/java/de/gsi/fcc/screenshot/app/util
MathUtil.java

$  cd ../common-utils 
$ git remote add repoA ../screenshot-app

$ git pull repoA master
warning: no common commits
remote: Counting objects: 36, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 36 (delta 1), reused 6 (delta 1)
Unpacking objects: 100% (36/36), done.
From ../screenshot-app
 * branch            master     -> FETCH_HEAD
Merge made by the 'recursive' strategy.
 src/main/java/de/gsi/fcc/screenshot/app/util/MathUtil.java | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
 create mode 100644 src/main/java/de/gsi/fcc/screenshot/app/util/MathUtil.java

$ ls src/main/java/de/gsi/fcc/screenshot/app/util 
MathUtil.java

$ git --no-pager log --oneline --decorate --graph --all -8
*   f2e8f00 (HEAD, master) Merge branch 'master' of ../screenshot-app
|\  
| * 7a6b4dd BP: Moved Callback class as ExceptionAwareRunnable to common-utils
| * fe36c33 BP: Run code cleanup
| * 1b6d6b0 BP: Migration to git. Changed ga(v) de.gsi.cs.co.ap.app -> de.gsi.fcc.applications, changed package names accordingly and updated common dependencies
* 187e43f (origin/master, origin/HEAD) BP: Added addIfNotPresent to GsiCollectionUtils
* ff19764 BP: Added ExceptionAwareRunnable
* 643e943 BP: Add BackgroundExceptionLogger
* 3b0ed16 JF: renamed groupId to de.gsi.fcc.commons and artifactID to common-utils

# Optional rebase, consider leaving the merge
$ git rebase origin/master 
First, rewinding head to replay your work on top of it...
Applying: BP: Migration to git. Changed ga(v) de.gsi.cs.co.ap.app -> de.gsi.fcc.applications, changed package names accordingly and updated common dependencies
Applying: BP: Run code cleanup
Applying: BP: Moved Callback class as ExceptionAwareRunnable to common-utils

$ git --no-pager log --oneline --decorate --graph --all -8
* 12e8b1e (HEAD, master) BP: Moved Callback class as ExceptionAwareRunnable to common-utils
* 1e80340 BP: Run code cleanup
* 6f03c47 BP: Migration to git. Changed ga(v) de.gsi.cs.co.ap.app -> de.gsi.fcc.applications, changed package names accordingly and updated common dependencies
* 187e43f (origin/master, origin/HEAD) BP: Added addIfNotPresent to GsiCollectionUtils
* ff19764 BP: Added ExceptionAwareRunnable
* 643e943 BP: Add BackgroundExceptionLogger
* 3b0ed16 JF: renamed groupId to de.gsi.fcc.commons and artifactID to common-utils
* 9119d44 GIT Migration from SVN: add .gitignore

#
# Then open the project in eclipse, see if it compiles, move it to a new package.
# In our example I moved the new files from  .../fcc/screenshot/app to .../cs/co/ap/common/utils
#

# Now add and commit those changes
#
git add .
git commit -m 'BP: Move MathUtil fcc.screenshot.app -> cs.co.ap.common.utils' 

#
# The files are now at the desired place including the history
#
git --no-pager log --oneline --decorate --graph --all -8                     
* e4b1708 (HEAD, master) BP: Move MathUtil fcc.screenshot.app -> cs.co.ap.common.utils
* 12e8b1e BP: Moved Callback class as ExceptionAwareRunnable to common-utils
* 1e80340 BP: Run code cleanup
* 6f03c47 BP: Migration to git. Changed ga(v) de.gsi.cs.co.ap.app -> de.gsi.fcc.applications, changed package names accordingly and updated common dependencies
* 187e43f (origin/master, origin/HEAD) BP: Added addIfNotPresent to GsiCollectionUtils
* ff19764 BP: Added ExceptionAwareRunnable
* 643e943 BP: Add BackgroundExceptionLogger
* 3b0ed16 JF: renamed groupId to de.gsi.fcc.commons and artifactID to common-utils

# You can push those now
git push

# Remove the project from eciplse
# And delete the temporary working directory
rm -rf ~/tmp/repomove

Result

Further Reading

• Delete files from history https://stackoverflow.com/questions/43762338/how-to-remove-file-from-git-history

GIT repository refactoring using git filter-repo

Project home

Prerequisites

Make sure our special GIT version is in the path. Clone git-filter-repo and make sure the git-filter-repo command is in your path.

export PATH=/common/home/bel/rmueller/lnx/local/opt/git/git-2.23.0/bin:$PATH
export PATH=/common/home/bel/YOUR_USERNAME/lnx/git/git-filter-repo:$PATH

Scenario 1 - Create reactor project from existing fat client fx application

1. Clone source repo (don't add to Eclipse, otherwise likely --force is required for git-filter-repo commands)
2. Create reactor repo based on fcc-webflux-javafx-template, initialize repo. More information on this archetype.
3. Remove sample files physically in reactor fx-app module.
4. Change paths in source repo

   git-filter-repo --path-rename :REACTOR_MODULE_FOLDER_NAME/ --refs master
   e.g. git-filter-repo --path-rename :bss-control-app-fx/ --refs master

5. In reactor repo, add source repo as remote

   git remote add fx-app-import SOURCE_REPO_FULL_LOCAL_PATH/.git

6. Integrate import branch into reactor repo

   git pull --no-rebase --allow-unrelated-histories fx-app-import master

7. Remove import remote from reactor repo

   git remote rm fx-app-import

8. Readd dependencies from sample app-fx pom.xml file to existing application pom.xml file now located in reactor repo:

   <dependency>
       <groupId>de.gsi.fcc.applications</groupId>
       <artifactId>webflux-uilib-fx</artifactId>
       <version>16.1.0-SNAPSHOT</version>
   </dependency>
   <dependency>
       <groupId>de.gsi.fcc.applications</groupId>
       <artifactId>APPNAME-client-lib</artifactId>
       <!-- e.g. <artifactId>bss-control-client-lib</artifactId> -->
       <version>16.1.0-SNAPSHOT</version>
   </dependency>