GitLab Object Storage Configuration

[xzt-13106431626]

GitLab has object storage enabled with proxy mode disabled. Currently, when uploading large files via the client using Git LFS, the returned URL address is the GitLab service address instead of the object storage URL.

GitLab configuration is as follows:

gitlab_rails['lfs_enabled'] = true
gitlab_rails['lfs_object_store_enabled'] = true
gitlab_rails['lfs_object_store_proxy_download'] = false
gitlab_rails['lfs_object_store_remote_directory'] = "gitlab-lfs"
gitlab_rails['lfs_object_store_connection'] = {
  'provider' => 'AWS',
  'region' => 'huhehaote1',
  'aws_access_key_id' => 'x',
  'aws_secret_access_key' => 'x',
  'endpoint' => 'https://eos-huhehaote-1.cmecloud.cn',
  'force_path_style' => true
}

### GitLab uploads
gitlab_rails['uploads_object_store_enabled'] = true
gitlab_rails['uploads_object_store_proxy_download'] = false
gitlab_rails['uploads_object_store_remote_directory'] = "gitlab-uploads"
gitlab_rails['uploads_object_store_connection'] = {
  'provider' => 'AWS',
  'region' => 'huhehaote1',
  'aws_access_key_id' => 'x',
  'aws_secret_access_key' => 'x',
  'endpoint' => 'https://eos-huhehaote-1.cmecloud.cn',
  'force_path_style' => true
}

Configuration validation shows it is effective:

root@gitlab-bf8dfc847-dvq5p:/# gitlab-rails console
--------------------------------------------------------------------------------
 Ruby:         ruby 3.1.4p223 (2023-03-30 revision 957bb7cb81) [x86_64-linux]
 GitLab:       16.9.1 (0ef8ba69a8f) FOSS
 GitLab Shell: 14.33.0
 PostgreSQL:   14.10
------------------------------------------------------------[ booted in 17.43s ]
Loading production environment (Rails 7.0.8)
irb(main):001:1* lfs_connection = Aws::S3::Client.new(
irb(main):002:1*   access_key_id:     'xxx',
irb(main):003:1*   secret_access_key: 'xx',
irb(main):004:1*   region:            'huhehaote1',
irb(main):005:1*   endpoint:          'https://eos-huhehaote-1.cmecloud.cn',
irb(main):006:0>   force_path_style:  true )
=> #<Aws::S3::Client>
irb(main):007:0> s3_resource = Aws::S3::Resource.new(client: lfs_connection)
=> #<Aws::S3::Resource:0x00007efdfb2b8b58 @client=#<Aws::S3::Client>>
irb(main):008:0> bucket = s3_resource.bucket('gitlab-lfs')
irb(main):009:0> puts bucket.exists?
true
=> nil
irb(main):010:0> # Upload file
=> nil
irb(main):011:0> bucket.object('test.txt').put(body: 'Test content')
irb(main):012:0* # Download file
irb(main):013:0> obj = bucket.object('test.txt')
irb(main):014:0> puts obj.get.body.read
Test content
=> nil

Error logs are as follows:

16:08:07.672334 trace git-lfs: tq: enqueue retry #8 after 10.00s for "d38637ef4e60ce027b574b4142cd639cce739ade469da65fa6d948f5216c841e" (size: 1914151)
16:08:17.672600 trace git-lfs: tq: sending batch of size 1
16:08:17.672753 trace git-lfs: api: batch 1 files
16:08:17.672787 trace git-lfs: HTTP: POST http://root:glpat-Lp8wFs6yMgWLs47CyGzb@192.168.10.11:30004/root/sss.git/info/lfs/objects/batch
...
16:08:17.719046 trace git-lfs: HTTP: {"objects":[{"oid":"d38637ef4e60ce027b574b4142cd639cce739ade469da65fa6d948f5216c841e","size":1914151,"actions":{"upload":{"href":"http://gitlab-bf8dfc847-dvq5p/root/sss.git/gitlab-lfs/objects/d38637ef4e60ce027b574b4142cd639cce739ade469da65fa6d948f5216c841e/1914151","header":{"Authorization":"Basic cm9vdDpleUowZVhBaU9pSktWMVFpTENKaGJHY2lPaUpJVXpJMU5pSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pY205dmRDSjlMQ0pxZEdraU9pSTVZamd3TURSbE55MDFaRGxtTFRRMFlUUXRPV1JoWXkwMk5ESXlaalJsWlRWaU9HRWlMQ0pwWVhRaU9qRTNOalF3TlRnd09UY3NJb
...
16:08:17.720041 trace git-lfs: HTTP: PUT http://gitlab-bf8dfc847-dvq5p/root/sss.git/gitlab-lfs/objects/d38637ef4e60ce027b574b4142cd639cce739ade469da65fa6d948f5216c841e/1914151
...
16:08:17.746184 trace git-lfs: tq: refusing to retry "d38637ef4e60ce027b574b4142cd639cce739ade469da65fa6d948f5216c841e", too many retries (8)
LFS: Put "http://gitlab-bf8dfc847-dvq5p/root/sss.git/gitlab-lfs/objects/d38637ef4e60ce027b574b4142cd639cce739ade469da65fa6d948f5216c841e/1914151": dial tcp: lookup gitlab-bf8dfc847-dvq5p on 114.114.114.114:53: no such host
Error: failed to push some refs to 'http://192.168.10.11:30004/root/sss.git'
[root@RK1 sss]#

What are the potential solutions to this issue?