HTTP ERROR 503 Service Unavailable

SCM-Manager
1

I hope someone can help me, it has been a bad 12 hours of debugging and I am not stuck on the 503 error.

I am running scm-manager on a proxmox ubuntu vm and it has been working great for several years. I use it for both SVN and GIT.

The issues all started when I was getting weird behavior when pushing new git branches and it turned out to be because the vm root file system was full (10G). The /var/lib/scm is mounted on a separate drive. I suspect it was log related because after adding more disk space (+10G) and rebooting, the usage dropped to 5G.

But that didn’t fix it. I tried upgrading to newest version, but that didnt help either. Finally I found that java was crashing due to HEAP memory error. I had 4G allocated. I kept growing the memory all the way to 16G, but it keep giving me HEAP out of memory. I suspected the reason was the content-search plugin, so I moved it out of the plugin folder and tried again.

It no longer crashes, but the web interface never comes available. I am normally reaching it through a rev-proxy. Both the https and the local http:8080/scm reports the 503 error.

The only warning in the log is shown below and honestly I am not sure what it means.

Can anyone offer any advice on how to resolve this. Help is appriciated.

Thanks

Sep 17 15:32:40 svn systemd[1]: Started SCM-Manager Server.
Sep 17 15:32:40 svn scm-server[1381]: SLF4J(W): No SLF4J providers were found.
Sep 17 15:32:40 svn scm-server[1381]: SLF4J(W): Defaulting to no-operation (NOP) logger implementation
Sep 17 15:32:40 svn scm-server[1381]: SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.
Sep 17 15:32:40 svn scm-server[1381]: Set http request header size to 16384
Sep 17 15:32:40 svn scm-server[1381]: Set http response header size to 16384
Sep 17 15:32:40 svn scm-server[1381]: Set forward request customizer: true
Sep 17 15:32:40 svn scm-server[1381]: Set http address binding to 0.0.0.0
Sep 17 15:32:40 svn scm-server[1381]: Set http port to 8080
Sep 17 15:32:40 svn scm-server[1381]: Set webapp war file to /opt/scm-server/var/webapp/scm-webapp.war
Sep 17 15:32:40 svn scm-server[1381]: Set webapp temp directory to /var/cache/scm/work/webapp
Sep 17 15:32:40 svn scm-server[1381]: Set webapp max form content size to 200000
Sep 17 15:32:40 svn scm-server[1381]: Set webapp max form keys to 1000
Sep 17 15:32:40 svn scm-server[1381]: Set docroot temp directory to /var/cache/scm/work/work/docroot
Sep 17 15:32:42 svn scm-server[1381]: 15:32:42.695 [main] INFO sonia.scm.lifecycle.classloading.ClassLoaderLifeCycle - create new simple ClassLoaderLifeCycle
Sep 17 15:32:42 svn scm-server[1381]: 15:32:42.702 [main] INFO sonia.scm.lifecycle.BootstrapContextFilter - register for restart events
Sep 17 15:32:42 svn scm-server[1381]: 15:32:42.704 [main] INFO sonia.scm.event.LegmanScmEventBus - create new event bus ScmEventBus-1
Sep 17 15:32:42 svn scm-server[1381]: 15:32:42.718 [main] DEBUG io.micrometer.common.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework
Sep 17 15:32:42 svn scm-server[1381]: 15:32:42.808 [main] INFO sonia.scm.event.ScmEventBus - use sonia.scm.event.LegmanScmEventBus as event bus implementation
Sep 17 15:32:43 svn scm-server[1381]: Writing logs to: /var/log/scm
Sep 17 15:32:46 svn scm-server[1381]: Sep 17, 2025 3:32:46 PM com.google.inject.servlet.GuiceFilter setPipeline
Sep 17 15:32:46 svn scm-server[1381]: WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate>

Scanning the log file did tell me much, the only none INFO is:

2025-09-17 20:23:55.825 [main] [          ] WARN  sonia.scm.cache.GuavaCacheConfigurationReader - could not find manual configuration at /var/lib/scm/ext/gcache.xml
2025-09-17 20:23:56.513 [main] [          ] WARN  sonia.scm.plugin.ExtensionBinder - could not find extension for extension point interface sonia.scm.api.v2.resources.LogoutRedirection
2025-09-17 20:23:56.552 [main] [          ] WARN  sonia.scm.plugin.ExtensionBinder - could not find extension for extension point interface sonia.scm.io.FileSystem
2025-09-17 20:23:58.942 [main] [          ] WARN  sonia.scm.cache.GuavaCacheConfigurationReader - could not find manual configuration at /var/lib/scm/ext/gcache.xml
2025-09-17 20:24:00.262 [main] [          ] WARN  io.micrometer.core.instrument.MeterRegistry - This FunctionCounter has been already registered (MeterId{name='executor.completed', tags=[tag(name=MustacheTemplateEngine),tag(type=cached)]}), the registration will be ignored. Note that subsequent logs will be logged at debug level.
2025-09-17 20:24:00.286 [main] [          ] WARN  sonia.scm.template.ServletMustacheFactory - could not find resource sonia/scm/mail/layout_en.mustache
2025-09-17 20:24:00.286 [main] [          ] WARN  sonia.scm.template.MustacheTemplateEngine - could not find mustache template at /sonia/scm/mail/layout_en.mustache
2

Hi @sten.sogaard , indeed this sounds bad. The logs you’ve included do not look suspicious. Are there any Exception entries? Besides, you could take a look at the xml files in the config directory. Are there any empty files or ones that are not properly closed? One more thing you can try is to delete your plugins directory or move it to another place. You don’t have to worry about your data, the plugins directories do not contain any data.
We’re keeping our fingers crossed!

3

I got it working but never found the root cause.

My solution was to build a new VM from scratch and install the same version of scm-manager. When I was sure it was working as it should, I started surgically transplanting files from /etc/scm/ and /var/lib/scm. I would compare the content of the yaml/xml files between the clean install and the old, and make edits if necessary, but for the repository folder I just moved it all.

I was able to restore the users, groups, permissions and repositories. It looks like it is working like it should. The client computers can now interact with the git and svn repos as if nothing happened.