Internal repository error with SVN repository

Hello,

I am using SCM-Manager 2.38.1 on Debian Linux 11.

While importing certain SVN repository backup, I constantly receive below error.

Log file has following entries for mentioned transaction id

2022-09-13 23:47:10.015 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.repository.DefaultRepositoryManager - create repository scmadmin/sarel.pazaryeri.entegrasyonu (3mTHLid8Y2r) of type svn
2022-09-13 23:47:10.022 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.io.DefaultFileSystem - create directory /var/lib/scm/repositories/3mTHLid8Y2r
2022-09-13 23:47:10.056 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.security.DefaultAuthorizationCollector - invalidate cache, because of a received authorization event
2022-09-13 23:47:10.163 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.importexport.FromBundleImporter - copied 19362368 bytes to temp, start bundle import
2022-09-13 23:47:10.164 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.repository.api.UnbundleCommandBuilder - unbundle archive /var/lib/scm/repositories/3mTHLid8Y2r/work/work-10540722046988683693/scm-import-12717774877916417956.bundle at scmadmin/sarel.pazaryeri.entegrasyonu (3mTHLid8Y2r)
2022-09-13 23:47:11.173 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.repository.DefaultRepositoryManager - delete repository scmadmin/sarel.pazaryeri.entegrasyonu (3mTHLid8Y2r) of type svn
2022-09-13 23:47:11.175 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.io.DefaultFileSystem - destroy directory /var/lib/scm/repositories/3mTHLid8Y2r
2022-09-13 23:47:11.185 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.security.DefaultAuthorizationCollector - invalidate cache, because of a received authorization event
  • Logout and login is not helping.
  • Restarting the scm-server is not helping.
  • SVN repository backup is from another SCM-Manager with identical version. ZIP file byte length is 19,362,368. Log file has entry that same bytes received.
  • Taking another backup and file byte size is same. That new backup is also giving same error.
  • There is no problem importing another repository even after having that error.

This is maybe the 40th repository that I am importing on same server and only this repository causing me troubles so far.

Administration - Subversion has following settings:
Version compatibility: No compatibility
Enable GZIP compression is checked

This SCM-Manager has only core plugins. No additional plugin is installed.

I appreciate any help fixing that error.

Thanks & Regards,
Ertan

Hello Ertan,

could you please check your log again. If there is a red error bar in the frontend, you should hopefully find a stack trace to the specific error somewhere in the log.

Regards, Eduard

Hi,

I do not have any ERROR lines in the log. I checked it more than once just to be sure before adding a new thread in here.

After your reply, I modified my logging details (it was installation default). I checked my final log line before starting repository import.

Below zip contains lines after import starts and stops when I get red error bar. I hope that would make sense to you as this is quite detailed.

scm_import.zip (9.8 KB)

It is possible, I did not increase logging detail for necessary parts. If so, just let me know what to increase detail and I can prepare another importing log part for you.

Thanks & Regards,
Ertan

The error says the Gnome Keyring Lib could not be found. Could you try to install it?

sudo apt-get install libgnome-keyring0

I just saw this lib is deprecated. Please find here more information:

Yes, libgnome-keyring0 package is not available in my system.

Link is a confirmation that libgnome-keyring package is deprecated.

I could not see a solution in the link though. Should I install libsecret-1-0 package instead and try?

Sure, try it out. If it does not help you, change your log level in the logging.xml to TRACE and send us the full log.

Thanks, Eduard

That didn’t help. Full TRACE log is here

scm-manager.zip (310.7 KB)

Thanks & Regards,
Ertan

Okay, this helped. The gnome-keyring error is not the reason your import fails.

This is the actual error:

Your dump files is kinda corrupt or at least not compatible. Could be again related to different/wrong encodings. Maybe find the solution here:

I see, but, I am not moving from Windows to Linux. Backup is taken on Linux Debian and restore is also handled on Linux Debian. Both systems use UTF8 locale. One is English (backup system), other is Turkish (restore system).

Backup is a ZIP file download from same version SCM-Manager.

I have more than 80 repositories. Only 4 of them failed. One is actually small 160KiB as ZIP file. I am willing to share that with you privately out of this forum if you would like to check it.

Correction. Both backup and restore Linux Debian systems have UTF8 Turkish encoding.

Hello,

I have checked my successfully imported backup file in raw. It does not have ':' for 'PROPS-END' like the error says. So, I believe this is not an encoding problem.

Here is partial dump beginning. This is RAW and no additional text in it.

SVN-fs-dump-format-version: 2

UUID: 5d95be6a-09b0-11ec-8c18-adbce46db3ce

Revision-number: 0
Prop-content-length: 56
Content-length: 56

K 8
svn:date
V 27
2021-08-30T16:36:03.109525Z
PROPS-END

Revision-number: 1
Prop-content-length: 131
Content-length: 131

K 7
svn:log
V 33
Creating_initial_branch_structure
K 10
svn:author
V 3
sve
K 8
svn:date
V 27
2021-08-30T16:36:03.292474Z
PROPS-END

Node-path: branches
Node-kind: dir
Node-action: add
Prop-content-length: 10
Content-length: 10

PROPS-END


Node-path: tags
Node-kind: dir
Node-action: add
Prop-content-length: 10
Content-length: 10

PROPS-END


Node-path: trunk
Node-kind: dir
Node-action: add
Prop-content-length: 10
Content-length: 10

PROPS-END


Revision-number: 2

As you can see, none of the PROPS-END lines have no ':' at the beginning. Since this dump is successfully imported by the SCM-Manager. Problem has to be somewhere else. Because, I have no ':' for any 'PROPS-END' in the failing backup either. My failing backup beginning is as following in RAW.

SVN-fs-dump-format-version: 2

UUID: 392169ee-09d3-11ec-a252-050a31fbf713

Revision-number: 0
Prop-content-length: 56
Content-length: 56

K 8
svn:date
V 27
2021-08-30T20:45:34.335941Z
PROPS-END

Revision-number: 1
Prop-content-length: 131
Content-length: 131

K 7
svn:log
V 33
Creating_initial_branch_structure
K 10
svn:author
V 3
sve
K 8
svn:date
V 27
2021-08-30T20:45:34.526854Z
PROPS-END

Node-path: branches
Node-kind: dir
Node-action: add
Prop-content-length: 10
Content-length: 10

PROPS-END


Node-path: tags
Node-kind: dir
Node-action: add
Prop-content-length: 10
Content-length: 10

PROPS-END


Node-path: trunk
Node-kind: dir
Node-action: add
Prop-content-length: 10
Content-length: 10

PROPS-END


Revision-number: 2
Prop-content-length: 96
Content-length: 96

K 7
svn:log
V 0

K 10
svn:author
V 2
ek
K 8
svn:date
V 27
2021-08-30T20:51:39.512319Z
PROPS-END

Is there anything else that I can check?

As a reminder, I have replaced my dump several times from identical version, identical OS, identical locale SCM-Manager and result is always the same.

Thanks & Regards,
Ertan

Hello @eheimbuch,

My case, I have 3 subversion GUI tools.

  • Old svn server which is retired now. Keeping it available until I am sure it is not needed anymore.
  • SCM-Manager_1 which has all repositories from my old server and all commits goes in this one at the moment.
  • SCM-Manager_2 which I am moving all repositories from my SCM-Manager_1 above. Once finished, this one will be actively used.

Old svn server and SCM-Manager_1 are on same server, SCM-Manager_2 is on separate server. All my servers are Debian Linux with same locale setting. Both SCM-Manager versions are identical and they are 2.39.0 as of now. All dumps are taken from SCM-Manager_1 until now as all commits actively goes here.

You were right saying that dump was corrupted.

I tried to import it using svnadmin command line and svnadmin claimed there is a problem with revision 8. I was lucky for this repository as my old svn server was at identical revision level. It was not modified a bit until now.

I checked my old svn server repository using svnadmin verify ./ and it was fine. I prepared a new dump file from that using svnadmin dump ./ > new.dump and that new dump successfully imported in SCM-Manager_2.

When I do “health check” for the repository on SCM-Manager_1 I do not get any error. When I try to use svnadmin verify on that repository I get below error

root@svn:/var/lib/scm/repositories/28TECpRbsEk# cat metadata.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<repositories>
    <properties/>
    <id>28TECpRbsEk</id>
    <namespace>scmadmin</namespace>
    <name>sarel.pazaryeri.entegrasyonu</name>
    <type>svn</type>
    <description></description>
    <contact></contact>
    <creationDate>1660222268482</creationDate>
    <lastModified>1660222278509</lastModified>
    <permission>
        <groupPermission>false</groupPermission>
        <name>scmadmin</name>
        <role>OWNER</role>
    </permission>
    <archived>false</archived>
</repositories>
root@svn:/var/lib/scm/repositories/28TECpRbsEk# svnadmin verify data/
svnadmin: E070014: Can't read file 'data/db/uuid': End of file found
root@svn:/var/lib/scm/repositories/28TECpRbsEk# cat data/db/uuid
392169ee-09d3-11ec-a252-050a31fbf713
root@svn:/var/lib/scm/repositories/28TECpRbsEk# ls -l data/db/uuid
-rw-r--r-- 1 scm scm 37 Ağu 11 15:51 data/db/uuid
root@svn:/var/lib/scm/repositories/28TECpRbsEk# whoami
root
root@svn:/var/lib/scm/repositories/28TECpRbsEk#

Comparing uuid file with old svn server and old one has same uuid value in two lines. SCM-Manager_1 has just one line. I manually add same uuid as second line and after that repository verification pass just fine.

root@svn:/var/lib/scm/repositories/28TECpRbsEk# svnadmin verify data
* Verifying repository metadata ...
* Verifying metadata at revision 2 ...
* Verifying metadata at revision 7 ...
* Verifying metadata at revision 8 ...
* Verifying metadata at revision 10 ...
* Verifying metadata at revision 12 ...
* Verifying metadata at revision 13 ...
* Verifying metadata at revision 17 ...
* Verifying metadata at revision 20 ...
* Verifying metadata at revision 21 ...
* Verifying metadata at revision 23 ...
* Verifying metadata at revision 28 ...
* Verifying metadata at revision 35 ...
* Verifying metadata at revision 37 ...
* Verified revision 0.
* Verified revision 1.
* Verified revision 2.
* Verified revision 3.
* Verified revision 4.
* Verified revision 5.
* Verified revision 6.
* Verified revision 7.
* Verified revision 8.
* Verified revision 9.
* Verified revision 10.
* Verified revision 11.
* Verified revision 12.
* Verified revision 13.
* Verified revision 14.
* Verified revision 15.
* Verified revision 16.
* Verified revision 17.
* Verified revision 18.
* Verified revision 19.
* Verified revision 20.
* Verified revision 21.
* Verified revision 22.
* Verified revision 23.
* Verified revision 24.
* Verified revision 25.
* Verified revision 26.
* Verified revision 27.
* Verified revision 28.
* Verified revision 29.
* Verified revision 30.
* Verified revision 31.
* Verified revision 32.
* Verified revision 33.
* Verified revision 34.
* Verified revision 35.
* Verified revision 36.
* Verified revision 37.
* Verified revision 38.
* Verified revision 39.
* Verified revision 40.
root@svn:/var/lib/scm/repositories/28TECpRbsEk#

Same uuid error exists for any repository that I try to run svnadmin verify. I think this is something to be checked for SCM-Manager.

Real problem here is having a dump on SCM-Manager_1 using svnadmin dump command line successfully imports on SCM-Manager_2.

However, having a dump on SCM-Manager_1 itself fails to be imported on SCM-Manager_2. I also tried to import that dump using svnadmin load and it also fails.

Let me know if I can be of any help for fixing these problems, please.

Thanks & Regards,
Ertan

Hey Ertan,

I have very limited time for community support since i am developing new paid features for our customers. Nonetheless we will try to help you with this import/export issue.

If the error really is related to creating dump files from your SCMM1 as you described you could backup your whole scm_home directory (all repositories) and try to upgrade this instance a newer SCMM2 version. Afterwards create the svn dump via the SCMM2 export function instead of using svnadmin. Maybe this will work for you. https://scm-manager.org/docs/2.39.x/en/user/repo/settings/

Migration guide: https://scm-manager.org/docs/2.39.x/en/migrate-scm-manager-from-v1/

PS: It is still odd that we got the error with the missing : for PROPS_END. Have you checked whether the broken dump and the working dumps have any difference in the dump header?

Hi,

I understand that your are busy. I am truly sorry about that.

What I was trying to explain is that under certain circumstances, SCM-Manager subversion backup is not usable and is corrupt.

I just wanted to provide enough information for you to have it figured and possibly fixed as that may also affect paid customers.

About comparison, it is not very easy as there are some data located at different lines in both files. Though, that is not a corruption and information is there and this is just fine.

However, there is a visible difference between SCM-Manager dump and svnadmin dump for content length numbers. Below example is from another repository which SCM-Manager dump and svnadmin dump is different

SCM-Manager dump header

Revision-number: 27
Prop-content-length: 218
Content-length: 218

K 7
svn:log
V 130
Logo E-Fatura için SHIPPING_AGENT etiketinde düzeltme yapıldı.
Logo kabul etmesi için başına sıfır eklemek gerkiyormuş.

K 10
svn:author
V 2
ek
K 8
svn:date
V 27
2022-06-18T13:53:41.846377Z
PROPS-END

svnadmin dump header

Revision-number: 27
Prop-content-length: 228
Content-length: 228

K 10
svn:author
V 2
ek
K 8
svn:date
V 27
2022-06-18T13:53:41.846377Z
K 7
svn:log
V 130
Logo E-Fatura için SHIPPING_AGENT etiketinde düzeltme yapıldı.
Logo kabul etmesi için başına sıfır eklemek gerkiyormuş.

PROPS-END

I checked another repository which has such problem and it is also the same, content length information is wrong.

It feels like this is because of UTF-8 characters. Because 10 character difference calculates like 10 UTF-8 characters which are displayed as 1 character but saved as 2. However, this cannot be the reason. Because, there are prior revisions in same SCM-Manager dump file which are actually correct and they do have similar UTF-8 characters written as 2 bytes.

On another dump, I observe that header data length written AFTER comment data is exactly same as the content length number difference. But, it is not the case for above example.

Problem is because content length is wrong in SCM-Manager dump. But, I cannot identify main reason of the problem. I only have some repositories dump by SCM-Manager which are broken.

On the other hand, there is no problem with svnadmin dumps. One difference I observe in svnadmin dumps is the header information is always written first, comment data after (mostly as last thing) in header. That is no corruption reason though.

You may need additional information, please let me know.

Thanks & Regards,
Ertan