Slow commits after initial run - Caching How to

Hello,

I am using SCM-Manager v2.38.1 on Debian Linux. Actually, I have more than one SCM-Manager.

Some of the systems are sometimes shutdown and restarted. First commit after a startup is always slow.

I am explained that there is a “caching” mechanism in order to speed this up a lot for this initial commit after restart.

However, I could not understand the document. Where to add and what to add clearly. Because settings files seems more complicated than single line parameters to me.

For example, self-signed certificate documentation https://scm-manager.org/docs/latest/en/administration/scm-server/ is clear for me. I could not understand clearly the above one.

Should I modify /etc/scm/scm-server.xml and add mentioned parameters in it? Is it another file that I should be adding parameters? What should be the final config file after parameter adding (XML structure is not clear)? There is a command line parameter but SCM-Manager runs on different linux distributions as well as windows. Some of them has different startup scripts. I am not quite sure where to add that command line parameter for my system Debian Linux. Should I modify my service startup scripts?

I really would like to have some more detailed instructions, please.

Any help is appreciated.

Thanks & Regards,
Ertan

Hi,

After some searching, I understand that system property is something specific to java.

For my Debian Linux, I had to define these properties directly in file /opt/scm-server/bin/scm-server This is an executable script which run SCM-Server.

I added parameters explained in documentation to EXTRA_JVM_ARGUMENTS value in the script as following

#!/bin/bash
# ----------------------------------------------------------------------------
#  Copyright 2001-2006 The Apache Software Foundation.
#
#  Licensed under the Apache License, Version 2.0 (the "License");
#  you may not use this file except in compliance with the License.
#  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
#  limitations under the License.
# ----------------------------------------------------------------------------

#   Copyright (c) 2001-2002 The Apache Software Foundation.  All rights
#   reserved.

# extra jvm arguments
EXTRA_JVM_ARGUMENTS="-Djava.awt.headless=true -Dlogback.configurationFile=logging.xml -Dscm.workingCopyPoolStrategy=sonia.scm.repository.work.SimpleCachingWorkingCopyPool"

BASEDIR="/opt/scm-server"

# load settings from defaults directory
[ -r /etc/default/scm-server ] && . /etc/default/scm-server

# If a specific java binary isn't specified search for the standard 'java' binary
if [ -z "$JAVACMD" ] ; then
  if [ -n "$JAVA_HOME"  ] ; then
    if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
      # IBM's JDK on AIX uses strange locations for the executables
      JAVACMD="$JAVA_HOME/jre/sh/java"
    else
      JAVACMD="$JAVA_HOME/bin/java"
    fi
  else
    JAVACMD=`which java`
  fi
fi

if [ ! -x "$JAVACMD" ] ; then
  echo "Error: JAVA_HOME is not defined correctly."
  echo "  We cannot execute $JAVACMD"
  exit 1
fi

if [ -z "$REPO" ]
then
  REPO="$BASEDIR"/lib
fi

CLASSPATH=$CLASSPATH_PREFIX:"$BASEDIR"/conf:"$REPO"/*

exec "$JAVACMD" $JAVA_OPTS \
  $EXTRA_JVM_ARGUMENTS \
  -classpath "$CLASSPATH" \
  -Dapp.name="scm-server" \
  -Dapp.pid="$$" \
  -Dapp.repo="$REPO" \
  -Dbasedir="$BASEDIR" \
  sonia.scm.server.ScmServerDaemon \
  "$@"

I am yet to check for its performance improvement.

I just wanted to save a solution here for others like me who have no idea what documentation says.

I am still trying to understand how to increase maximum capacity of the cache from default 5 repositories.

Thanks & Regards,
Ertan

Hi Ertan,

thank you for contributing and sharing your solution with the community.

That is really appreciated!

Please let us know if your solution will improve the performance. Probalby this is something we can incorporate in your official documentation.

Best,
Christoph

Hey Ertan,

Maybe you are looking for “scm.workingCopyPoolSize”?

Example: scm/scm-server at afb4921702e7c1bbb92f9f7c2fc772860ac9a302 · cloudogu/scm · GitHub

Regards, Eduard

Hi Eduard,

Probably this is what I was searching for. Without understanding it much your given code lines, I added the parameter and set it to 10. Now the relevant line is as follows

EXTRA_JVM_ARGUMENTS="-Djava.awt.headless=true -Dlogback.configurationFile=logging.xml -Dscm.workingCopyPoolSize=10 -Dscm.workingCopyPoolStrategy=sonia.scm.repository.work.SimpleCachingWorkingCopyPool"

I still cannot clearly understand how caching works.

  • Documentation indicates a maximum capacity. This capacity as I understand is the repository count (it is called clone) to be cached. Is that correct?
  • Since we did not set any config parameter with any repository name, how this caching selects which repositories to be saved in cache? For example, initial start of the server for the first time after setting these parameters, is there anything automatically cached?

Thanks & Regards,
Ertan

If i am not mistaken this caching strategy is for keeping repository clones (like you mentioned). We keep the “x” latest clones which were used internally. If we perform any kind of content changes inside a repository, for example merges via the review plugin or simple file edit via the editor plugin, we work with internal clones. Depending on the repository size it can decrease the performance to check out these clones every time. This is the reason we added this caching mechanism.

The pool size should define how many clones are being kept in the internal working directories of your SCM-Manager. If you keep too many clones you could potentially get issues with your disk space, so our default pool size is “5”. If you are interested to more details you may check our documentation near the implementation: scm-manager/SimpleCachingWorkingCopyPool.java at 13d72d45c64907026ffb9b885c1c68599ff81012 · scm-manager/scm-manager · GitHub

Hope this could help you,
Eduard

Hi Christoph,

It has been sometime I am using caching on two servers. My experience is as follows:

I did get a good response time on server1. Speed increase is so much visible.
Server2, I did not get that much of a speed increase.

They both are Linux Debian servers with identical CPUs. Server1 has larger 8 GB RAM and server2 has 3GB only.

My guess is that speed increase linked to certain amount of free memory.

Thanks & Regards,
Ertan

1 Like

I am closing this topic as there has been no activity for more than 30 days.