Internal repository error with SVN repository

ertank · September 13, 2022, 9:12pm

Hello,

I am using SCM-Manager 2.38.1 on Debian Linux 11.

While importing certain SVN repository backup, I constantly receive below error.

Log file has following entries for mentioned transaction id

2022-09-13 23:47:10.015 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.repository.DefaultRepositoryManager - create repository scmadmin/sarel.pazaryeri.entegrasyonu (3mTHLid8Y2r) of type svn
2022-09-13 23:47:10.022 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.io.DefaultFileSystem - create directory /var/lib/scm/repositories/3mTHLid8Y2r
2022-09-13 23:47:10.056 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.security.DefaultAuthorizationCollector - invalidate cache, because of a received authorization event
2022-09-13 23:47:10.163 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.importexport.FromBundleImporter - copied 19362368 bytes to temp, start bundle import
2022-09-13 23:47:10.164 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.repository.api.UnbundleCommandBuilder - unbundle archive /var/lib/scm/repositories/3mTHLid8Y2r/work/work-10540722046988683693/scm-import-12717774877916417956.bundle at scmadmin/sarel.pazaryeri.entegrasyonu (3mTHLid8Y2r)
2022-09-13 23:47:11.173 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.repository.DefaultRepositoryManager - delete repository scmadmin/sarel.pazaryeri.entegrasyonu (3mTHLid8Y2r) of type svn
2022-09-13 23:47:11.175 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.io.DefaultFileSystem - destroy directory /var/lib/scm/repositories/3mTHLid8Y2r
2022-09-13 23:47:11.185 [qtp1031980531-123] [1dTHLiOSLdV] INFO  sonia.scm.security.DefaultAuthorizationCollector - invalidate cache, because of a received authorization event

Logout and login is not helping.
Restarting the scm-server is not helping.
SVN repository backup is from another SCM-Manager with identical version. ZIP file byte length is 19,362,368. Log file has entry that same bytes received.
Taking another backup and file byte size is same. That new backup is also giving same error.
There is no problem importing another repository even after having that error.

This is maybe the 40th repository that I am importing on same server and only this repository causing me troubles so far.

Administration - Subversion has following settings:
Version compatibility: No compatibility
Enable GZIP compression is checked

This SCM-Manager has only core plugins. No additional plugin is installed.

I appreciate any help fixing that error.

Thanks & Regards,
Ertan

eheimbuch · September 14, 2022, 6:53am

Hello Ertan,

could you please check your log again. If there is a red error bar in the frontend, you should hopefully find a stack trace to the specific error somewhere in the log.

Regards, Eduard

ertank · September 14, 2022, 8:48am

Hi,

I do not have any ERROR lines in the log. I checked it more than once just to be sure before adding a new thread in here.

After your reply, I modified my logging details (it was installation default). I checked my final log line before starting repository import.

Below zip contains lines after import starts and stops when I get red error bar. I hope that would make sense to you as this is quite detailed.

scm_import.zip (9.8 KB)

It is possible, I did not increase logging detail for necessary parts. If so, just let me know what to increase detail and I can prepare another importing log part for you.

Thanks & Regards,
Ertan

eheimbuch · September 14, 2022, 9:12am

The error says the Gnome Keyring Lib could not be found. Could you try to install it?

sudo apt-get install libgnome-keyring0

eheimbuch · September 14, 2022, 9:14am

I just saw this lib is deprecated. Please find here more information:

ertank · September 14, 2022, 9:20am

Yes, libgnome-keyring0 package is not available in my system.

Link is a confirmation that libgnome-keyring package is deprecated.

I could not see a solution in the link though. Should I install libsecret-1-0 package instead and try?

eheimbuch · September 14, 2022, 9:23am

Sure, try it out. If it does not help you, change your log level in the logging.xml to TRACE and send us the full log.

Thanks, Eduard

ertank · September 14, 2022, 9:39am

That didn’t help. Full TRACE log is here

scm-manager.zip (310.7 KB)

Thanks & Regards,
Ertan

eheimbuch · September 14, 2022, 9:47am

Okay, this helped. The gnome-keyring error is not the reason your import fails.

This is the actual error:

Your dump files is kinda corrupt or at least not compatible. Could be again related to different/wrong encodings. Maybe find the solution here:

ertank · September 14, 2022, 9:54am

I see, but, I am not moving from Windows to Linux. Backup is taken on Linux Debian and restore is also handled on Linux Debian. Both systems use UTF8 locale. One is English (backup system), other is Turkish (restore system).

Backup is a ZIP file download from same version SCM-Manager.

I have more than 80 repositories. Only 4 of them failed. One is actually small 160KiB as ZIP file. I am willing to share that with you privately out of this forum if you would like to check it.

ertank · September 14, 2022, 10:02am

Correction. Both backup and restore Linux Debian systems have UTF8 Turkish encoding.

ertank · September 15, 2022, 2:12pm

Hello,

I have checked my successfully imported backup file in raw. It does not have ':' for 'PROPS-END' like the error says. So, I believe this is not an encoding problem.

Here is partial dump beginning. This is RAW and no additional text in it.

SVN-fs-dump-format-version: 2

UUID: 5d95be6a-09b0-11ec-8c18-adbce46db3ce

Revision-number: 0
Prop-content-length: 56
Content-length: 56

K 8
svn:date
V 27
2021-08-30T16:36:03.109525Z
PROPS-END

Revision-number: 1
Prop-content-length: 131
Content-length: 131

K 7
svn:log
V 33
Creating_initial_branch_structure
K 10
svn:author
V 3
sve
K 8
svn:date
V 27
2021-08-30T16:36:03.292474Z
PROPS-END

Node-path: branches
Node-kind: dir
Node-action: add
Prop-content-length: 10
Content-length: 10

PROPS-END


Node-path: tags
Node-kind: dir
Node-action: add
Prop-content-length: 10
Content-length: 10

PROPS-END


Node-path: trunk
Node-kind: dir
Node-action: add
Prop-content-length: 10
Content-length: 10

PROPS-END


Revision-number: 2

As you can see, none of the PROPS-END lines have no ':' at the beginning. Since this dump is successfully imported by the SCM-Manager. Problem has to be somewhere else. Because, I have no ':' for any 'PROPS-END' in the failing backup either. My failing backup beginning is as following in RAW.

SVN-fs-dump-format-version: 2

UUID: 392169ee-09d3-11ec-a252-050a31fbf713

Revision-number: 0
Prop-content-length: 56
Content-length: 56

K 8
svn:date
V 27
2021-08-30T20:45:34.335941Z
PROPS-END

Revision-number: 1
Prop-content-length: 131
Content-length: 131

K 7
svn:log
V 33
Creating_initial_branch_structure
K 10
svn:author
V 3
sve
K 8
svn:date
V 27
2021-08-30T20:45:34.526854Z
PROPS-END

Node-path: branches
Node-kind: dir
Node-action: add
Prop-content-length: 10
Content-length: 10

PROPS-END


Node-path: tags
Node-kind: dir
Node-action: add
Prop-content-length: 10
Content-length: 10

PROPS-END


Node-path: trunk
Node-kind: dir
Node-action: add
Prop-content-length: 10
Content-length: 10

PROPS-END


Revision-number: 2
Prop-content-length: 96
Content-length: 96

K 7
svn:log
V 0

K 10
svn:author
V 2
ek
K 8
svn:date
V 27
2021-08-30T20:51:39.512319Z
PROPS-END

Is there anything else that I can check?

As a reminder, I have replaced my dump several times from identical version, identical OS, identical locale SCM-Manager and result is always the same.

Thanks & Regards,
Ertan

ertank · September 15, 2022, 10:34pm

Hello @eheimbuch,

My case, I have 3 subversion GUI tools.

Old svn server which is retired now. Keeping it available until I am sure it is not needed anymore.
SCM-Manager_1 which has all repositories from my old server and all commits goes in this one at the moment.
SCM-Manager_2 which I am moving all repositories from my SCM-Manager_1 above. Once finished, this one will be actively used.

Old svn server and SCM-Manager_1 are on same server, SCM-Manager_2 is on separate server. All my servers are Debian Linux with same locale setting. Both SCM-Manager versions are identical and they are 2.39.0 as of now. All dumps are taken from SCM-Manager_1 until now as all commits actively goes here.

You were right saying that dump was corrupted.

I tried to import it using svnadmin command line and svnadmin claimed there is a problem with revision 8. I was lucky for this repository as my old svn server was at identical revision level. It was not modified a bit until now.

I checked my old svn server repository using svnadmin verify ./ and it was fine. I prepared a new dump file from that using svnadmin dump ./ > new.dump and that new dump successfully imported in SCM-Manager_2.

When I do “health check” for the repository on SCM-Manager_1 I do not get any error. When I try to use svnadmin verify on that repository I get below error

root@svn:/var/lib/scm/repositories/28TECpRbsEk# cat metadata.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<repositories>
    <properties/>
    <id>28TECpRbsEk</id>
    <namespace>scmadmin</namespace>
    <name>sarel.pazaryeri.entegrasyonu</name>
    <type>svn</type>
    <description></description>
    <contact></contact>
    <creationDate>1660222268482</creationDate>
    <lastModified>1660222278509</lastModified>
    <permission>
        <groupPermission>false</groupPermission>
        <name>scmadmin</name>
        <role>OWNER</role>
    </permission>
    <archived>false</archived>
</repositories>
root@svn:/var/lib/scm/repositories/28TECpRbsEk# svnadmin verify data/
svnadmin: E070014: Can't read file 'data/db/uuid': End of file found
root@svn:/var/lib/scm/repositories/28TECpRbsEk# cat data/db/uuid
392169ee-09d3-11ec-a252-050a31fbf713
root@svn:/var/lib/scm/repositories/28TECpRbsEk# ls -l data/db/uuid
-rw-r--r-- 1 scm scm 37 Ağu 11 15:51 data/db/uuid
root@svn:/var/lib/scm/repositories/28TECpRbsEk# whoami
root
root@svn:/var/lib/scm/repositories/28TECpRbsEk#

Comparing uuid file with old svn server and old one has same uuid value in two lines. SCM-Manager_1 has just one line. I manually add same uuid as second line and after that repository verification pass just fine.

root@svn:/var/lib/scm/repositories/28TECpRbsEk# svnadmin verify data
* Verifying repository metadata ...
* Verifying metadata at revision 2 ...
* Verifying metadata at revision 7 ...
* Verifying metadata at revision 8 ...
* Verifying metadata at revision 10 ...
* Verifying metadata at revision 12 ...
* Verifying metadata at revision 13 ...
* Verifying metadata at revision 17 ...
* Verifying metadata at revision 20 ...
* Verifying metadata at revision 21 ...
* Verifying metadata at revision 23 ...
* Verifying metadata at revision 28 ...
* Verifying metadata at revision 35 ...
* Verifying metadata at revision 37 ...
* Verified revision 0.
* Verified revision 1.
* Verified revision 2.
* Verified revision 3.
* Verified revision 4.
* Verified revision 5.
* Verified revision 6.
* Verified revision 7.
* Verified revision 8.
* Verified revision 9.
* Verified revision 10.
* Verified revision 11.
* Verified revision 12.
* Verified revision 13.
* Verified revision 14.
* Verified revision 15.
* Verified revision 16.
* Verified revision 17.
* Verified revision 18.
* Verified revision 19.
* Verified revision 20.
* Verified revision 21.
* Verified revision 22.
* Verified revision 23.
* Verified revision 24.
* Verified revision 25.
* Verified revision 26.
* Verified revision 27.
* Verified revision 28.
* Verified revision 29.
* Verified revision 30.
* Verified revision 31.
* Verified revision 32.
* Verified revision 33.
* Verified revision 34.
* Verified revision 35.
* Verified revision 36.
* Verified revision 37.
* Verified revision 38.
* Verified revision 39.
* Verified revision 40.
root@svn:/var/lib/scm/repositories/28TECpRbsEk#

Same uuid error exists for any repository that I try to run svnadmin verify. I think this is something to be checked for SCM-Manager.

Real problem here is having a dump on SCM-Manager_1 using svnadmin dump command line successfully imports on SCM-Manager_2.

However, having a dump on SCM-Manager_1 itself fails to be imported on SCM-Manager_2. I also tried to import that dump using svnadmin load and it also fails.

Let me know if I can be of any help for fixing these problems, please.

Thanks & Regards,
Ertan

eheimbuch · September 16, 2022, 6:58am

Hey Ertan,

I have very limited time for community support since i am developing new paid features for our customers. Nonetheless we will try to help you with this import/export issue.

If the error really is related to creating dump files from your SCMM1 as you described you could backup your whole scm_home directory (all repositories) and try to upgrade this instance a newer SCMM2 version. Afterwards create the svn dump via the SCMM2 export function instead of using svnadmin. Maybe this will work for you. https://scm-manager.org/docs/2.39.x/en/user/repo/settings/

Migration guide: https://scm-manager.org/docs/2.39.x/en/migrate-scm-manager-from-v1/

PS: It is still odd that we got the error with the missing : for PROPS_END. Have you checked whether the broken dump and the working dumps have any difference in the dump header?

ertank · September 16, 2022, 9:16am

Hi,

I understand that your are busy. I am truly sorry about that.

What I was trying to explain is that under certain circumstances, SCM-Manager subversion backup is not usable and is corrupt.

I just wanted to provide enough information for you to have it figured and possibly fixed as that may also affect paid customers.

About comparison, it is not very easy as there are some data located at different lines in both files. Though, that is not a corruption and information is there and this is just fine.

However, there is a visible difference between SCM-Manager dump and svnadmin dump for content length numbers. Below example is from another repository which SCM-Manager dump and svnadmin dump is different

SCM-Manager dump header

Revision-number: 27
Prop-content-length: 218
Content-length: 218

K 7
svn:log
V 130
Logo E-Fatura iÃ§in SHIPPING_AGENT etiketinde dÃ¼zeltme yapÄ±ldÄ±.
Logo kabul etmesi iÃ§in baÅŸÄ±na sÄ±fÄ±r eklemek gerkiyormuÅŸ.

K 10
svn:author
V 2
ek
K 8
svn:date
V 27
2022-06-18T13:53:41.846377Z
PROPS-END

svnadmin dump header

Revision-number: 27
Prop-content-length: 228
Content-length: 228

K 10
svn:author
V 2
ek
K 8
svn:date
V 27
2022-06-18T13:53:41.846377Z
K 7
svn:log
V 130
Logo E-Fatura iÃ§in SHIPPING_AGENT etiketinde dÃ¼zeltme yapÄ±ldÄ±.
Logo kabul etmesi iÃ§in baÅŸÄ±na sÄ±fÄ±r eklemek gerkiyormuÅŸ.

PROPS-END

I checked another repository which has such problem and it is also the same, content length information is wrong.

It feels like this is because of UTF-8 characters. Because 10 character difference calculates like 10 UTF-8 characters which are displayed as 1 character but saved as 2. However, this cannot be the reason. Because, there are prior revisions in same SCM-Manager dump file which are actually correct and they do have similar UTF-8 characters written as 2 bytes.

On another dump, I observe that header data length written AFTER comment data is exactly same as the content length number difference. But, it is not the case for above example.

Problem is because content length is wrong in SCM-Manager dump. But, I cannot identify main reason of the problem. I only have some repositories dump by SCM-Manager which are broken.

On the other hand, there is no problem with svnadmin dumps. One difference I observe in svnadmin dumps is the header information is always written first, comment data after (mostly as last thing) in header. That is no corruption reason though.

You may need additional information, please let me know.

Thanks & Regards,
Ertan

christoph.loose · October 18, 2022, 2:14pm

Hi @ertank

we tried in several ways (like set encoding to turkish and mixed it with english and many more) to reproduce your issues but still without any success.

This is the point where we have to say that on one hand it isn’t reproducable and on the other hand we would need a lot of time to make progress.

Therefore we are asking the community.
Do you have experienced similar issues? Do you have ideas what caused this issue?

christoph.loose · December 27, 2022, 9:40am

I am closing this topic as there has been no activity for more than 30 days.

Topic		Replies	Views
SVN-Error E730053 SCM-Manager	13	783	June 12, 2023
Could not get content from revision SCM-Manager	5	365	December 27, 2022
SCM-Manager 2.34.0 SCM-Manager release	5	296	May 23, 2022
Error mirroring the SVN repositories Bug Reports SCM-Manager	10	402	June 16, 2022
Trouble committing after import of repository from 1.6.0 Bug Reports SCM-Manager	3	291	April 24, 2023

Internal repository error with SVN repository

Related Topics