Skip to content

Conversation

@billygor
Copy link
Contributor

this change improves drastically pre-push behaviour, by not sending
lfs objects which are already on a remote. Works perfectly with
pushing new branches and tags.

currently pre-push command analyse "local sha1" vs "remote sha1" of the
ref being pushed and if "remote sha1" is available locally tries to send
only lfs objects introduced with new commits.

why this is broken:

  • remote branch might have moved forward (local repo is not up to date).
    In this case you have no chance to isolate new lfs objects ("remote sha1"
    does not exist locally) and git-lfs sends everything from the local
    branch history.
  • remote branch does not exist (or new tag is pushed). Same consequences.

But what is important - local repository always have remote references,
from which user created his local branch and started making some local
changes. So, all we have to do is to identify new lfs objects which do
not exist on remote references. And all this can be easily achieved with
the same all mighty git rev-list command.

This change makes git-lfs usable with gerrit, where changes are uploaded
by using magic gerrit branches which does not really exist. i.e.
git push origin master:refs/for/master

in this case "refs/for/master" does not exist and git feeds all 0-s as
"remote sha1".

@technoweenie
Copy link
Contributor

I like this 🤘 Would like at least 1 more review though. Should be able to merge it early next week. Thanks!

lfs/scanner.go Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick but it's not always scanning two refs, it can scan one (refRight == "") and in fact that's the more common usage. I'd call it ScanRefs or something similar instead.

@sinbad
Copy link
Contributor

sinbad commented Aug 14, 2015

👍 except for my nitpick suggestions. Together with the recently merged #581 this should make pre-push much better.

@technoweenie
Copy link
Contributor

👍 on the nitpicks. How about these names: ScanRefs, ScanAll, ScanLeftToRemote?

@billygor billygor force-pushed the major-pre-push-optimization-for-current-remote branch from db234a9 to 08c8ffe Compare August 17, 2015 08:54
this change improves drastically pre-push behaviour, by not sending
lfs objects which are already on a remote. Works perfectly with
pushing new branches and tags.

currently pre-push command analyse "local sha1" vs "remote sha1" of the
ref being pushed and if "remote sha1" is available locally tries to send
only lfs objects introduced with new commits.

why this is broken:
- remote branch might have moved forward (local repo is not up to date).
  In this case you have no chance to isolate new lfs objects ("remote sha1"
  does not exist locally) and git-lfs sends everything from the local
  branch history.
- remote branch does not exist (or new tag is pushed). Same consequences.

But what is important - local repository always have remote references,
from which user created his local branch and started making some local
changes. So, all we have to do is to identify new lfs objects which do
not exist on remote references. And all this can be easily achieved with
the same all mighty git rev-list command.

This change makes git-lfs usable with gerrit, where changes are uploaded
by using magic gerrit branches which does not really exist. i.e.
git push origin master:refs/for/master

in this case "refs/for/master" does not exist and git feeds all 0-s as
"remote sha1".
@billygor billygor force-pushed the major-pre-push-optimization-for-current-remote branch from 08c8ffe to f3bf4ea Compare August 17, 2015 08:59
@billygor
Copy link
Contributor Author

rebased and uploaded new version.
@sinbad, thanks. I've strongly typed ScanMode.
As for adding "refRight" to "ScanDeltaToRemote" mode: In pre-push scenario I consider 3 options:

  1. refRight is all zeros (means we are pushing new ref) - no need to include refRight.
  2. local repo is up to date, which means we have refRight as one of remotes refs and there is no effect of adding refRight to git rev-list --objects <left> --not --remotes="remote name"
  3. local repo is not up to date, ref on a remote is pointing to some commit we don't have yet, and so we cannot add refRight to git rev-list, it will return with failure.

May be I'm missing something here. If you'll explain me a scenario where this might be still needed, I can add a check that "refRight" exists locally (git cat-file -e <sha1> - will do), and add it to git rev-list if it does. And I would prefer to do it with a subsequent commit.

@technoweenie, I took your naming advice, but ScanRefs name is already used as the function name. Had to add "Mode" to ScanMode names.

@sinbad
Copy link
Contributor

sinbad commented Aug 17, 2015

You're right that in the current push use case you don't need to support left & right & remotes, but ScanRefs is a general utility function so is not limited to that use case in future. I can't say there will never be a use case for 'everything between 2 commits that also isn't on remote'. I don't mind that use case not being catered for but the signature should be clear that supplying the right ref will do nothing. [edit]Which now you've renamed it to ScanLeftToRemoteMode it does, so I'm fine with that[/edit]

@rubyist
Copy link
Contributor

rubyist commented Aug 17, 2015

👍 from me, looks rad

technoweenie added a commit that referenced this pull request Aug 17, 2015
…-current-remote

major pre-push optimization (alternative: look for deltas against current remote only)
@technoweenie technoweenie merged commit 4457d7c into git-lfs:master Aug 17, 2015
@billygor billygor deleted the major-pre-push-optimization-for-current-remote branch August 18, 2015 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants