Sap bc performance test

Udo Paltzer
IST

SAP BC 4.7 Performance Test
June 2003


Table of Contents:

1 Introduction ....................................................................................................................3

2 Performance test tool and infrastructure .....................................................................3
2.1 Test tool .......................................................................................................................................................3

2.2 Infrastructure...............................................................................................................................................3

3 Test cases and results in detail ....................................................................................4
3.1 Introduction .................................................................................................................................................4
3.1.1 Volume series...........................................................................................................................................4
3.1.2 Scalability under parallel clients ...............................................................................................................5

3.2 stfc_performance ........................................................................................................................................5
3.2.1 Volume series...........................................................................................................................................5
3.2.2 Scalability under parallel clients ...............................................................................................................7
3.2.3 Performance Tuning...............................................................................................................................12

3.3 XSLT ...........................................................................................................................................................17

3.4 bapisdorder_getdetailedlist .....................................................................................................................19

3.5 IDoc-XML....................................................................................................................................................19
3.5.1 Volume series.........................................................................................................................................19
3.5.2 Scalability under parallel clients .............................................................................................................20

3.6 EDI ..............................................................................................................................................................23
3.6.1 Scalability under parallel clients .............................................................................................................23

3.7 HTML page.................................................................................................................................................24

3.8 Java performance test..............................................................................................................................25

4 Summary of tests results.............................................................................................26
4.1 Volume series............................................................................................................................................27
4.1.1 Varying incoming XML document size ...................................................................................................27
4.1.2 Varying the response RFC size .............................................................................................................27

4.2 Scalability under parallel clients .............................................................................................................27

4.3 Performance Tuning .................................................................................................................................27

4.4 Java performance test..............................................................................................................................28

4.5 Comparison with SAP BC 4.6 performance measurements.................................................................28

5 Copyright ......................................................................................................................29

SAPBC47_PerformanceTests.doc Page 2


1 Introduction
This is a SAP Business Connector (SAP BC) 4.7 performance test report produced from the SAP BC
development group at SAP. Its goal is to provide customers and partners of SAP with some reliable sample
data on the scalability and throughput you can expect using SAP BC Release 4.7. A comparison of the
performance on different platforms and with different Java Virtual machines is not the goal of this report.
All scenarios tested in this report are inbound scenarios where SAP BC is always called synchronously via a
XML document.

2 Performance test tool and infrastructure

2.1 Test tool
For all tests the freely available tool WebBench 4.1 by Ziff Davis Inc. has been used. It was chosen over
WebStone (Mindcraft, Inc.) and MS WebCapacity Analysis Tool (Microsoft) because of its comfortable GUI
and the various possibilities to customize the test cases. WebBench consists of two components: a controller
instance and distributed clients which connect to the HTTP server to test. The clients themselves may start
several threads to simulate parallel requests. The controller drives and monitors the clients. At the beginning
of each test the test parameters are submitted to the clients, at the end of the test run the results are
collected from the clients and a performance report is generated.
The CPU and memory load of the participating test machines was monitored with the MS Performance
Monitor and the MS Task Manager.
The performance of the SAP basis systems involved was monitored post mortem via transaction STAT.
Further checking was executed via RFC tracing (transaction ST05) and the gateway monitor (transaction
SMGW).

2.2 Infrastructure
The hardware used has the following technical data:
Processors: 4 x Pentium II Xeon
Frequency: 400 MHz
Memory: 2 GB
L2 – Cache: 512 KB
Operating System: Windows NT 4.0, Build 1381
Network Card: 100 MBit
The different performance tests have been carried out on one of two machines with these technical data.
One computer was a HP (formerly Compaq) Proliant Server, which will be called computer A from now on,
and the other machine was a HP Netserver LH4, which will be called computer B from now on. The
performance of the system configuration in this SAP environment corresponds to about 600 SAPS (more
information about SAPS can be found at http://www.sap.com/benchmark -> SAPS).
The same machine is used to run the performance test HTTP clients as well as SAP BC. The advantage is
that the performance test is not influenced by the speed of the network between the HTTP clients and the
SAP BC server. Whether the SAP instance is running locally or remotely does not have a noticeable effect
on the test results however. This is due to the fact, that the data transferred is highly redundant and thus
compressed by the RFC very efficiently. So the amount of data transferred between SAP BC server and SAP
application server is always small.
All performance measurements were carried out with version 4.7 of SAP BC. SAP BC is running under IBM’s
JDK 1.3.0. The startup script server.bat is modified to allow the JVM a maximum memory usage of 2048
MB. Also the number of allowed parallel RFC connections to one SAP system is increased to 25.
Furthermore the throughput data were not logged and the log level was set to 1. The log level of the XSLT



engine has been set to 3. Last but not least in most of the test scenarios the parameters for optimizing the
performance have been set to the following values (except in these test cases, where the values of these
parameters have been changed in order to investigate their influence on the performance of SAP BC):
• watt.PartnerMgr.xtn.store.fastAsyncMode = True
• watt.PartnerMgr.xtn.store.timeToLive = 100
• watt.PartnerMgr.noMsgStorage = True
Tests against a SAP system were executed with the SAP kernel release 4.6C. All executed scenarios are
SAP-inbound. This means that SAP BC is always called from a HTTP client. It then transforms this call into
an RFC call to a backend SAP system. The HTTP client posts a XML request document to SAP BC. Along
with the HTTP header no cookie is transferred. A cookie would guarantee reentrance of several HTTP
requests to the same SAP BC session. However the performance impact of reentrance over the creation of
new sessions is minimal.
For all test scenarios we measure the number of completed requests and responses per second as well as
the total amount of data transferred per second. Furthermore, the SAP BC 4.7 test results are compared with
the measurements for SAP BC 4.6. The SAP BC 4.6 performance test report with all the details can be found
on the SAP Service Marketplace at http://service.sap.com/connectors -> SAP Business Connector -> SAP
Business Connector in Detail -> System Requirements.

3 Test cases and results in detail

3.1 Introduction
For most of the test scenarios volume tests as well as multi threading tests have been executed. The test
scenarios can be roughly categorized as follws:
• Call of the function module stfc_performance
• Transfer of an invoice IDoc XML document
• Transfer of an invoice EDI document (only multi threading tests)
The test scenarios with the call of the bapisdorder_getdetailedlist have not been carried out since an SAP
System with application data similar to the SAP BC 4.6 performance tests was not available. However, since
the SAP BC services invoked in these scenarios have not been changed, it can be assumed that the
performance measurements of SAP BC 4.7 and SAP BC 4.6 lead to almost the same results. Furthermore,
tests with SAP BC 4.7 for the bapisdorder_getdetailedlist scenarios lead to the same behaviour as the SAP
BC 4.6 test results, i. e. in the volume tests the SAP BC 4.7 processing time per request increases almost
linearly with increasing response document size, the throughput time decreases almost exponentially with
increasing response document size, and in the multi threading tests the saturation is reached with about 6
concurrent clients while increasing the number of clients further leads to only a small increase or even a
decrease in performance.
In one test scenario the parameters for optimizing the performance have been investigated. In another test
scenario the performance of a SAP BC flow map was compared with the performance of the XSLT engine in
SAP BC.
Furthermore a transaction which consists of a pure "HTTP Get" to an HTML page was executed (only multi
threading tests). Last but not least a Java performance test program was executed that carried out several
operations, e. g. adding and multiplication of numbers, throws of exceptions and writes to the file system;
hereby the operations were called via SAP BC Java services.

3.1.1 Volume series
These tests serve to determine the scalability of SAP BC with varying document size. A single client
connects to the server and as soon as it returns the next request is submitted. Therefore these scenarios
involve strictly sequential processing and only use one thread. By parallizing requests a higher throughput is
reachable. All 4 server CPUs are used for the volume tests.



Depending on the input values the SAP function module returns a varying amount of data as an internal
table. It makes a difference whether you submit a big XML document to the server and receive only a
minimal response or whether a small request document is submitted and a big response received from the
SAP system. This is due to two facts: First parsing of the XML document with the parser used by the RFC
coder takes longer than rendering an XML document from the RFC data format. Second the creation of a big
output table in the SAP system takes considerable time. Therefore both scenarios were tested separately.
3.1.1.1 Varying incoming XML document size
A business case for this scenario is data replication. Another are outbound calls where a small request
document is sent out and a bigger response document is returned from an external application.
To test this with the performance test tool XML documents of varying sizes are submitted to SAP BC via
"HTTP Post". The input size covers the range between 1 KB and 864 KB. For input sizes bigger than 1 MB
the HTTP performance test client will terminate with an error. Therefore tests with larger input documents
could not be performed.
3.1.1.2 Varying the response RFC size
The business case for this scenario is a standard inbound call. Based on a few input parameters a bigger
response is generated and returned from the SAP system.
These tests are performed by posting a small XML document (between 1 KB and 45 KB) to the server. An
input parameter determines the size of the internal table which is generated and returned by the function
module. The sizes ranged from 2 KB to 11.4 MB. As the time spent in the SAP system is not negligible here
it is measured separately. By subtracting the SAP time from the total runtime a virtual SAP BC response rate
and a virtual SAP BC throughput rate is calculated.

3.1.2 Scalability under parallel clients
These tests simulate a business scenario where several clients access the server at the same time.
For comparability all the clients use one of the workload files which is also used during the volume tests. The
number of parallel requests (which simulate HTTP clients) was increased from 1 to 16. Reduction of the
number of parallel requests after an overload situation does not show any hysteresis. Rather the
performance is in total accordance with the performance measured before the server is hit by the maximum
number of parallel clients. For the multi threadings tests all 4 server CPUs were used.

3.2 stfc_performance

3.2.1 Volume series
3.2.1.1 Varying incoming XML document size
These tests are performed by posting RFC XML documents of varying sizes to SAP BC via "HTTP Post".
The different input sizes are the following:
• 2 KB
• 12 KB
• 109 KB
• 217 KB
• 540 KB
• 864 KB
The tests were performed on computer A (for details see chapter 2.2).
The results for this test scenario are shown in the following graph:



The rounded numbers of the test results are summarized in the following table:

Input XML Document Seconds / Request Throughput
Size [KB] (Seconds / KB)
2 0.02 0.04
12 0.03 0.06
109 0.11 0.20
217 0.24 0.43
540 0.67 1.20
864 1.14 2.05

The SAP BC processing time per request increases almost linearly with increasing input document size. The
gradient is about 0.0013 with an offset of about 0. Also the throughput time increases almost linearly with
increasing input document size.
The performance measurements of SAP BC 4.7 and SAP BC 4.6 for this scenario lead to almost the same
results. This is due to the fact that the SAP BC services invoked in this scenario have not been changed in
SAP BC 4.7.
3.2.1.2 Varying the response RFC size
These tests are performed by posting a small RFC XML document (1 KB) to the server. An input parameter
determines the size of the internal table which is generated and returned by the function module. The
different response sizes are the following:
• 2 KB
• 12 KB
• 115 KB
• 229 KB
• 571 KB
• 1142 KB
• 5704 KB
• 11407 KB





Response Document Size Seconds / Request Throughput
[KB] (Seconds / MB)
2 0.04 22.4
12 0.04 3.42
115 0.08 0.74
229 0.14 0.61
571 0.27 0.48
1142 0.53 0.48
5704 2.22 0.40
11407 4.61 0.42

The SAP BC processing time per request increases almost linearly with increasing response document size.
The gradient is about 0.0004 with an offset of about 0.0366. The throughput time decreases almost
exponentially with increasing response document size.
SAP BC 4.7.

These tests are performed by posting a RFC XML document of the same size to SAP BC. The numbers of
parallel clients used are as follows:
• 1
• 2
• 4



• 6
• 8
• 10
• 12
• 16
3.2.2.1 10 KB document sizes
These tests are performed by posting a small RFC XML document (12 KB for inbound and 1 KB for
outbound (12 KB response document document size)) to SAP BC.
3.2.2.1.1 Inbound scenario


Parallel Clients Milli Seconds / Request Throughput
(Seconds / MB)
1 31.6 58.2
2 17.9 32.9
4 11.9 21.9
6 11.3 20.9
8 11.6 21.3
10 10.7 19.7
12 11.4 21.0
16 12.9 23.6

The saturation of the 4 processor SAP BC server is reached with about 6 concurrent clients. Increasing the
number of clients further leads to only a small increase or even a decrease in performance in this particular
performance test environment. This is an artifact due to the way the performance test is setup: the HTTP
clients run on the same host and each one eats up processing time. When the SAP BC is restricted to two or



one CPUs and the performance test client is restricted to another CPU you see that the performance
remains constant in an overload situation.
With an input document size of 12 KB you can expect to process more than 93 requests per second
(including SAP time) with a 4 CPU 400 MHz server. This corresponds to a number of more than 334800
requests which can be processed per hour for this scenario.
SAP BC 4.7.
3.2.2.1.2 Outbound scenario


(Seconds / MB)
1 40.3 3.47
2 22.7 1.95
4 14.8 1.28
6 12.0 1.03
8 12.6 1.08
10 12.6 1.09
12 11.8 1.01
16 12.9 1.10




With a response document size of 12 KB you can expect to process about 85 requests per second (including
SAP time) with a 4 CPU 400 MHz server. This corresponds to a number of about 306000 requests which can
be processed per hour for this scenario.
SAP BC 4.7.
These tests are performed by posting a medium size RFC XML document (109 KB for inbound and 1 KB for
outbound (115 KB response document document size)) to SAP BC.
3.2.2.2.1 Inbound scenario


(Seconds / MB)
1 108.7 200.0
2 59.5 109.5
4 42.4 77.9
6 36.2 66.7
8 37.6 69.2
10 35.7 65.7
12 37.3 68.6
16 37.3 68.6




With an input document size of 109 KB you can expect to process about 28 requests per second (including
SAP time) with a 4 CPU 400 MHz server. This corresponds to a number of about 100800 requests which can
be processed per hour for this scenario.
SAP BC 4.7.
3.2.2.2.2 Outbound scenario


(Seconds / MB)
1 83.3 0.75
2 46.7 0.42
4 27.2 0.24
6 24.0 0.22
8 22.7 0.20
10 23.5 0.21
12 22.2 0.20
16 22.7 0.20

number of clients further leads to only a small increase in performance in this particular performance test
environment. This is an artifact due to the way the performance test is setup: the HTTP clients run on the
same host and each one eats up processing time. When the SAP BC is restricted to two or one CPUs and
the performance test client is restricted to another CPU you see that the performance remains constant in an
overload situation.
With a response document size of 115 KB you can expect to process more than 45 requests per second



SAP BC 4.7.

3.2.3 Performance Tuning
In these test cases the following parameters for optimizing the performance have been investigated:
• watt.PartnerMgr.noMsgStorage (This switch can be set to "true" or "false". If set to "true", then the
message body of the incoming document will not be stored to disk, although a transaction will be
created (or maintained) for the incoming document, and the transaction can be monitored later on in
the transaction list.)
• watt.PartnerMgr.xtn.store.fastAsyncMode (This switch can take the values "true" or "false". If set to
"true", then the information in the message store will be read or written asynchronously.)
• watt.PartnerMgr.xtn.store.timeToLive (The switch can be used to set the "Time-To-Live" threshold
value of the asynchronous read/write cache. It is only effective when "fastAsyncMode" is set to true.
The "Time-To-Live" value determines how many seconds a transaction will be kept in the internal
cache until it gets purged.)
Hereby the following combinations of these parameters have been investigated:
• Scenario 1: noMsgStorage = false, fastAsyncMode = false
• Scenario 2: noMsgStorage = true, fastAsyncMode = false
• Scenario 3: noMsgStorage = false, fastAsyncMode = true, timeToLive = 100
• Scenario 4: noMsgStorage = true, fastAsyncMode = true, timeToLive = 100
3.2.3.1 Volume series
These tests are performed by posting RFC XML documents of varying sizes to SAP BC which processed the
documents via tRFC to the SAP system. The different input sizes are the following:
• 3 KB
• 12 KB
• 109 KB
• 217 KB
• 541 KB
• 864 KB
The tests were performed on computer B (for details see chapter 2.2).
For this scenario 8 parallel clients and all 4 server CPUs were used.
The results for these test scenarios are shown in the following graphs:




Scenario 1 Scenario 2 Scenario 3 Scenario 4
Input Sec. / Through Sec. / Through Sec. / Through Sec. / Through
XML Req. put Req. put Req. put Req. put
Doc. (Sec. / (Sec. / (Sec. / (Sec. /
Size MB) MB) MB) MB)
[KB]
3 0.03 17.83 0.02 12.41 0.03 16.26 0.02 12.16
12 0.04 3.21 0.03 2.16 0.04 2.99 0.03 2.13
109 0.09 0.77 0.08 0.68 0.08 0.76 0.06 0.57
217 0.16 0.73 0.10 0.46 0.16 0.71 0.10 0.46



Input Sec. / Through Sec. / Through Sec. / Through Sec. / Through
XML Req. put Req. put Req. put Req. put
Doc. (Sec. / (Sec. / (Sec. / (Sec. /
Size MB) MB) MB) MB)
[KB]
541 0.44 0.79 0.29 0.53 0.43 0.78 0.30 0.53
864 1.11 1.25 0.77 0.87 1.11 1.25 0.79 0.89

The following table summarizes the SAP BC processing time per request for the scenarios 1, 2, 3 in ratio to
the SAP BC processing time per request for the scenario 4, which shows the best performance results:

Input Ratio Scenario 1 to Ratio Scenario 2 to Ratio Scenario 3 to
XML Scenario 4 Scenario 4 Scenario 4
Doc.
Size
[KB]
3 1.5 1.0 1.3
12 1.5 1.0 1.4
109 1.3 1.2 1.3
217 1.6 1.0 1.5
541 1.5 1.0 1.5
864 1.4 1.0 1.4

In all 4 test scenarios the SAP BC processing time per request increases almost linearly with increasing input
document size for document sizes up to about 500 KB. For larger input document sizes the increase in SAP
BC processing time per request is bigger and significantly depends on the parameter settings for optimizing
the performance. The throughput time decreases almost exponentially with increasing input document size in
all 4 test scenarios.
The best performance can be achieved with the parameter settings of scenarios 4 and 2, i. e. the parameter
watt.PartnerMgr.noMsgStorage is set to true and thus the message body is not stored to the file system. The
parameter setting “watt.PartnerMgr.noMsgStorage = true” improves the performance of especially large
documents and also safes disc space. The influence on the performance of the parameter settings
watt.PartnerMgr.xtn.store.fastAsyncMode and watt.PartnerMgr.xtn.store.timeToLive is very small and can be
almost neglected.
The performance of scenarios 1 and 3, where the parameter watt.PartnerMgr.noMsgStorage is set to false,
is much slower than in scenarios 2 and 4, especially for large documents. This is due to the fact that in
scenarios 1 and 3 the message body is stored to the file system and that is time consuming, especially for
the bigger documents. The influence on the performance of the parameter settings
watt.PartnerMgr.xtn.store.fastAsyncMode and watt.PartnerMgr.xtn.store.timeToLive is very small and can be
almost neglected.
For all 4 scenarios the test results of SAP BC 4.7 show a performance increase compared with the
measurements of SAP BC 4.6. The performance increase is especially large for scenarios 1 and 3, and the
ratio of the SAP BC 4.6 processing time per request to the SAP BC 4.7 processing time per request reaches
up to a factor of 4 for large documents. For scenarios 2 and 4 the performance increase is also especially
high for large documents, and the ratio of the SAP BC 4.6 processing time per request to the SAP BC 4.7
processing time per request reaches up to a factor of 1.5.
3.2.3.2 Scalability under parallel clients



These tests are performed by posting a small RFC XML document (12 KB) to SAP BC. The numbers of
• 1
• 2
• 4
• 6
• 8
• 10
• 12
• 16
The results for these test scenarios are shown in the following graphs:




Parallel Milli Through Milli Through Milli Through Milli Through
Clients Sec. / put Sec. / put Sec. / put Sec. / put
Req. (Sec. / Req. (Sec. / Req. (Sec. / Req. (Sec. /
MB) MB) MB) MB)
1 116.3 9.87 81.97 6.95 113.6 9.64 76.92 6.53
2 65.79 5.58 40.32 3.42 56.82 4.82 40.32 3.42
4 41.32 3.51 27.62 2.34 35.71 3.03 24.88 2.11
6 36.76 3.12 25.00 2.12 34.48 2.93 24.27 2.06
8 38.46 3.26 25.00 2.12 37.31 3.17 25.25 2.14
10 34.72 2.95 26.18 2.22 34.25 2.91 27.32 2.32
12 37.31 3.17 27.03 2.29 34.25 2.91 24.63 2.09
16 34.01 2.89 25.25 2.14 34.01 2.89 24.51 2.08

The following table summarizes the SAP BC processing time per request for the scenarios 1, 2, 3 in ratio to
the SAP BC processing time per request for the scenario 4, which shows the best performance results:

Parallel Ratio Scenario 1 to Ratio Scenario 2 to Ratio Scenario 3 to
Clients Scenario 4 Scenario 4 Scenario 4
1 1.51 1.07 1.48
2 1.63 1.00 1.41
4 1.66 1.11 1.44
6 1.51 1.03 1.42
8 1.52 0.99 1.48



Parallel Ratio Scenario 1 to Ratio Scenario 2 to Ratio Scenario 3 to
Clients Scenario 4 Scenario 4 Scenario 4
10 1.27 0.96 1.25
12 1.51 1.10 1.39
16 1.39 1.03 1.39

In all 4 test scenarios the saturation of the 4 processor SAP BC server is reached with about 6 concurrent
clients. Increasing the number of clients further leads to only a small increase or even a decrease in
performance in this particular performance test environment. This is an artifact due to the way the
performance test is setup: the HTTP clients run on the same host and each one eats up processing time.
When the SAP BC is restricted to two or one CPUs and the performance test client is restricted to another
CPU you see that the performance remains constant in an overload situation.
The SAP BC processing time per request depends on the parameter settings for optimizing the performance.
However the performance does not depend on the parameter settings as significantly as in the volume
series, only for one HTP client. This is due to the fact that only a small document was posted to SAP BC (for
details see the performance results of the volume series in chapter 3.2.3.1).
Like in the volume series the best performance of the scalability tests can be achieved with the parameter
settings of scenarios 4 and 2, i. e. the parameter watt.PartnerMgr.noMsgStorage is set to true and thus the
message body is not stored to the file system. The parameter setting “watt.PartnerMgr.noMsgStorage = true”
improves the performance of especially large documents and also safes disc space. The influence on the
performance of the parameter settings watt.PartnerMgr.xtn.store.fastAsyncMode and
watt.PartnerMgr.xtn.store.timeToLive is very small and can be almost neglected.
Also like in the volume series the slowest performance of the scalability tests can be achieved with the
scenarios 1 and 3, where the parameter watt.PartnerMgr.noMsgStorage is set to false. This is due to the fact
that in scenarios 1 and 3 the message body is stored to the file system and that is time consuming. The
influence on the performance of the parameter settings watt.PartnerMgr.xtn.store.fastAsyncMode and
watt.PartnerMgr.xtn.store.timeToLive is very small and can be almost neglected, especially for a large
number of concurrent clients.
For all 4 scenarios the test results of SAP BC 4.7 show a performance increase compared with the
measurements of SAP BC 4.6. The performance increase is especially large for scenario 2, and the ratio of
the SAP BC 4.6 processing time per request to the SAP BC 4.7 processing time per request reaches up to a
factor of 2.9. For scenario 1 the ratio of the SAP BC 4.6 processing time per request to the SAP BC 4.7
processing time per request reaches up to a factor of 2.0, and for scenario 3 the ratio of the SAP BC 4.6
processing time per request to the SAP BC 4.7 processing time per request reaches up to a factor of 1.4.

3.3 XSLT
Two test scenarios have been carried out in order to compare the performance of a SAP BC flow map with
the performance of the XSLT engine in SAP BC. The transformation defined in the map and by the XSLT
Stylesheet is the same. The transformation contains only one step in which data are mapped into strings.
The data used in the transformation are received by posting a small RFC XML document (1 KB) to the SAP
BC server in order to call the function module stfc_performance. An input parameter determines the size of
the internal table which is generated and returned by the function module. The different response sizes are
the following:
• 2 KB
• 12 KB
• 115 KB
• 229 KB
• 571 KB
• 1142 KB



• 5704 KB
• 11407 KB


Response Document Seconds / Request Seconds / Request Ratio XSLT to
Size [KB] with XSLT with Map Flow MAP
2 0.08 0.06 1.30
12 0.10 0.07 1.47
115 0.25 0.09 2.78
229 0.43 0.12 3.57
571 1.00 0.19 5.35
1142 2.00 0.33 6.10
5704 10.0 1.33 7.50
11407 30.3 2.61 11.6

The SAP BC processing time per request for the flow map scenario increases almost linearly with increasing
response document size. The SAP BC processing time per request for the XSLT scenario increases almost
linearly for response document sizes up to about 6000 KB and increases at a higher rate for larger response
document sizes. Furthermore the SAP BC processing time per request of the flow map scenario is smaller
than the SAP BC processing time per request of the XSLT scenario for all response document sizes. Since
the ratio of the SAP BC processing time per request of the XSLT scenario to the SAP BC processing time
per request of the flow map scenario increases with increasing response document size, the performance for
the flow map scenario is better than the performance for the XSLT scenario, especially for large response
document sizes.
The performance measurements of SAP BC 4.7 and SAP BC 4.6 for the flow map scenario lead to almost
the same results. For the XSLT scenario the test results of SAP BC 4.7 show a performance increase



compared with the measurements of SAP BC 4.6. The performance increase is mainly due to a new version
of the XSLT engine within SAP BC 4.7. The ratio of the SAP BC 4.6 processing time per request to the SAP
BC 4.7 processing time per request reaches up to a factor of 2 for large documents. Thus the ratio of the
SAP BC 4.7 processing time per request of the XSLT scenario to the SAP BC 4.7 processing time per
request of the flow map scenario is up to a factor of 2 smaller than ratio of the SAP BC 4.6 processing time
per request of the XSLT scenario to the SAP BC 4.6 processing time per request of the flow map scenario.

3.4 bapisdorder_getdetailedlist
The test scenarios with the call of the bapisdorder_getdetailedlist have not been carried out since an SAP
System with application data similar to the SAP BC 4.6 performance tests was not available. However, since
the SAP BC services invoked in these scenarios have not been changed, it can be assumed that the
performance measurements of SAP BC 4.7 and SAP BC 4.6 lead to almost the same results. Furthermore,
tests with SAP BC 4.7 for the bapisdorder_getdetailedlist scenarios lead to the same behaviour as the SAP
BC 4.6 test results, i. e. in the volume tests the SAP BC 4.7 processing time per request increases almost
linearly with increasing response document size, the throughput time decreases almost exponentially with
increasing response document size, and in the multi threading tests the saturation is reached with about 6
concurrent clients while increasing the number of clients further leads to only a small increase or even a
decrease in performance.

3.5 IDoc-XML

3.5.1 Volume series
For this scenario only the incoming XML document size was altered, not the response RFC. The tests are
performed by posting IDoc XML documents of varying sizes to SAP BC via "HTTP Post". The different input
sizes are the following:

Input XML Document IDoc Segments RFC Size [KB]
Size [KB]
9 55 59
21 165 177
48 429 460
98 913 978
193 1837 1965





Input XML Document Seconds / Request Throughput
Size [KB] (Seconds / KB)
9 0.28 4.74
21 0.40 6.83
48 0.77 13.13
98 1.67 28.44
193 3.33 56.89

The SAP BC processing time per request increases almost linearly with increasing input document size. The
gradient is about 0.0169 with an offset of about 0.0443. Also the throughput time increases almost linearly
with increasing input document size.
For this scenario the test results of SAP BC 4.7 show a slight performance increase compared with the
measurements of SAP BC 4.6. The ratio of the SAP BC 4.6 processing time per request to the SAP BC 4.7

These tests are performed by posting an IDoc XML document of the same size to SAP BC. The numbers of
• 1
• 2
• 6
• 4
• 8
• 10
• 12
• 16



These tests are performed by posting a small IDoc XML document (9 KB) to SAP BC.


Parallel Clients Seconds / Request Throughput
(Seconds / KB)
1 0.26 4.38
2 0.15 2.55
4 0.10 1.63
6 0.08 1.38
8 0.08 1.32
10 0.08 1.37
12 0.08 1.43
16 0.08 1.34

With an input document size of 9 KB you can expect to process more than 12 requests per second (including
SAP time) with a 4 CPU 400 MHz server. This corresponds to a number of more than 43200 requests which
can be processed per hour for this scenario.
For this scenario the test results of SAP BC 4.7 show a performance increase compared with the



These tests are performed by posting a medium size IDoc XML document (98 KB) to SAP BC.


Parallel Clients Seconds / Request Throughput
(Seconds / KB)
1 1.25 21.33
2 0.71 12.19
4 0.45 7.76
6 0.34 5.89
8 0.32 5.51
10 0.30 5.17
12 0.27 4.61
16 0.27 4.61

number of clients further leads to only a small increase in performance in this particular performance test
environment. This is an artifact due to the way the performance test is setup: the HTTP clients run on the
same host and each one eats up processing time. When the SAP BC is restricted to two or one CPUs and
the performance test client is restricted to another CPU you see that the performance remains constant in an
overload situation.
With an input document size of 98 KB you can expect to process more than 3 requests per second (including
SAP time) with a 4 CPU 400 MHz server. This corresponds to a number of more than 10800 requests which
can be processed per hour for this scenario.



3.6 EDI

For this scenario only multi threading tests have been investigated. The tests are performed by posting a
small EDI document (35 KB) to SAP BC. SAP BC transferred the EDI document into an IDoc and processed
it to the SAP system. The numbers of parallel clients used are as follows:
• 1
• 2
• 4
• 6
• 8


Parallel Clients Seconds / Request
1 14.93
2 7.52
4 5.0
6 4.0



Parallel Clients Seconds / Request
8 3.75

number of clients further leads to a decrease in performance in this particular performance test environment
(not shown in the table and graph). This is an artifact due to the way the performance test is setup: the HTTP
With an input document size of 35 KB you can expect to process more than 0.26 requests per second
processing time per request reaches up to a factor of 1.8 for large documents.

3.7 HTML page
A transaction which consists of a pure "HTTP Get" to an HTML page was executed. The HTML page is
installed in a SAP BC package's pub directory and has the size of 1 KB. In this scenario SAP BC acts as a
mere HTTP server. The server returns an HTML page with a simple "Hello World" string. For this scenario
only multi threading tests have been investigated. The numbers of parallel clients used are as follows:
• 1
• 2
• 4
• 6
• 8
• 10
• 12
• 16




(Seconds / KB)
1 12.06 0.33
2 6.94 0.19
4 5.32 0.15
6 5.57 0.15
8 5.77 0.16
10 5.95 0.16
12 5.91 0.16
16 5.87 0.16

number of clients further leads to a decrease in performance in this particular performance test environment.
This is an artifact due to the way the performance test is setup: the HTTP clients run on the same host and
each one eats up processing time. When the SAP BC is restricted to two or one CPUs and the performance
test client is restricted to another CPU you see that the performance remains constant in an overload
situation.
With an input document size of 1 KB you can expect to process more than 187 requests per second with a 4
CPU 400 MHz server. This corresponds to a number of more than 673200 requests which can be processed
per hour for this scenario.
SAP BC 4.7.

3.8 Java performance test
A Java performance test program was executed that carried out several operations in a loop; hereby the
operations were called via SAP BC Java services. The tests were performed on computer A as well as on
computer B (for details see chapter 2.2). The executed operations, the number of iterations, the rounded
numbers of the test results, and the performance time ratio for both computers are summarized in the
following table.

Operation Number of Time [Seconds] Time [Seconds] Time Ratio
Iterations Computer A Computer B Computer A to
Computer B
Iteration of empty 10 million 0.000 0.000 1
loop
Addition of 10 million 0.047 0.063 0.75
integers in a loop
(0 + 1 + 2 + …)
Multiplication of 10 million 0.093 0.094 0.99
integers in a loop
(1 * 2 * 3 * …)
Addition of doubles 10 million 0.078 0.078 1



Operation Number of Time [Seconds] Time [Seconds] Time Ratio
Iterations Computer A Computer B Computer A to
Computer B
(db) in a loop
(db + db + db + …)
Multiplication of 10 million 2.875 2.875 1
doubles (db) in a
loop
(db * db * db * …)
Assignement of 1 million 0.016 0.032 0.50
integers to array of
length 1 million
Access to object 10 million 0.062 0.062 1
integer field
Method calls in the 10 million 0.094 0.094 1
same object
Method calls in 10 million 0.109 0.109 1
another object
Throw and catch of 10 million 11.563 11.672 0.99
exceptions
3 threads, 10000 0.047 0.047 1
switches for each
thread to stop CPU
access
Writes of 1 byte to 1 million 8.094 9.469 0.85
a file
Write of 1 MB to a 1 0.281 0.032 8.8
file
Cumulative 23.359 24.627 0.95
runtime

With this Java performance test program the biggest performance difference between both computers is
found for the write of 1 MB to a file; here computer B is about 8.8 times faster than computer A. The second
largest performance difference shows the assignement of integers to an array; here computer A is about 2
times faster than computer B. The third biggest performance difference is found for the addition of integers in
a loop; here computer A is about 25 per cent times faster than computer B. The fourth biggest performance
difference shows the writes of 1 byte to a file; here computer A is about 15 per cent times faster than
computer B. For the remaining Java performance tests the results for both computers are very similar; the
deviation is about 1 per cent or less.
Note: For these time measurements functionalities of the Java Virtual Machine are used which have an
accuracy of about 15 to 16 milli seconds. Therefore the processing time measurements in above table reflect
the steps of 15 and 16 milli seconds accuracy.
SAP BC 4.7.

4 Summary of tests results
The performance measurements show that in general the scalability of SAP BC is very good. The reasons
are summarized in the following sections of this chapter. At large document sizes or with many concurrent



client calls the tested scenarios become limited by the processing power of the SAP BC server. Memory of
the server has not been a restriction. The SAP system's performance was constant even under a high
parallel load but is very sensitive to the number of statements executed in the called function module.

4.1 Volume series

4.1.1 Varying incoming XML document size
In all test scenarios the SAP BC processing time per request as well as the throughput time increase almost
linearly with increasing input document size.

4.1.2 Varying the response RFC size
In all test scenarios the SAP BC processing time per request increases almost linearly with increasing
response document size. The throughput time decreases almost exponentially with increasing response
document size. However the SAP BC processing time per request for the XSLT scenario increases almost
linearly for response document sizes up to about 6000 KB and increases at a higher rate for larger response
document sizes. The performance for the flow map scenario is better than the performance for the XSLT
scenario, especially for large response document sizes.

4.2 Scalability under parallel clients
In most test scenarios the saturation of the 4 processor SAP BC server is reached with about 6 concurrent
clients. Increasing the number of clients further leads to a only small increase or even a decrease in
performance in this particular performance test environment. This is an artifact due to the way the
performance test is setup: the HTTP clients run on the same host and each one eats up processing time.
When the SAP BC is restricted to two or one CPUs and the performance test client is restricted to another
CPU you see that the performance remains constant in an overload situation for most test scenarios.
The following table summarizes the multi threading results and describes for all test scenarios, what SAP BC
processing time per request can be expected and how many requests per hour including SAP time (except
for the HTML page scenario, no SAP system is involved in this test case) can be processed approximately
for a particular input or response document size and number of CPUs used.

Test Scenario Input Response CPUs Milli Requests /
Document Document Seconds / Hour
Size [KB] Size [KB] Request
stfc_performance 12 4 10.7 334800
IDoc-XML 9 4 77.5 43200
IDoc-XML 98 4 270.3 10800
EDI 35 4 3745 936
HTML page 1 4 5.32 673200

4.3 Performance Tuning
The best performance in the volume series tests as well as in the scalability scenarios can be achieved when
the parameters watt.PartnerMgr.noMsgStorage and watt.PartnerMgr.xtn.store.fastAsyncMode are both set to
true. The parameter setting “watt.PartnerMgr.noMsgStorage = true” improves the performance of especially



large documents and also safes disc space, because the message body is not stored to the file system. With
the parameter watt.PartnerMgr.noMsgStorage set to true the influence on the performance of the parameter
settings watt.PartnerMgr.xtn.store.fastAsyncMode and watt.PartnerMgr.xtn.store.timeToLive is in the volume
series tests as well as in the scalability tests very small and can be almost neglected.
The slowest performance in the volume series tests as well as in the scalability scenarios can be achieved
when the parameters watt.PartnerMgr.noMsgStorage and watt.PartnerMgr.xtn.store.fastAsyncMode are both
set to false. With these parameter settings the message body is stored to the file system and the information
in the message store is updated synchronously, which is time consuming, and thus leads to the slowest
performance results. With the parameter watt.PartnerMgr.noMsgStorage set to false the influence on the
performance of the parameter settings watt.PartnerMgr.xtn.store.fastAsyncMode and
watt.PartnerMgr.xtn.store.timeToLive is in the volume series tests as well as in the scalability tests very small
and can be almost neglected, too.

4.4 Java performance test
For most of the performance tests executed with a Java performance test program the results of both
computers are very similar; the deviation is about 1 per cent or less. The biggest performance difference
between both computers is found for the write of 1 MB to a file; here computer B is about 8.8 times faster
than computer A. The second largest performance difference shows the assignement of integers to an array;
here computer A is about 2 times faster than computer B. The third biggest performance difference is found
for the addition of integers in a loop; here computer A is about 25 per cent times faster than computer B.

4.5 Comparison with SAP BC 4.6 performance measurements
The performance measurements of SAP BC 4.7 and SAP BC 4.6 lead to almost the same results for those
scenarios, in which SAP BC services are invoked that have not been changed in SAP BC 4.7. The
scenarios, which lead to almost the same test results, are the HTML page scenario, the Java performance
tests, and the call of the function module stfc_performance except for the performance tuning and the XSLT
scenarios. For the remaining scenarios the test results of SAP BC 4.7 show a performance increase
compared with the measurements of SAP BC 4.6.
For the performance tuning scenarios, i. e. the asynchronous call of the function module stfc_performance,
the performance increase is especially large for the volume series scenarios, in which the parameter
watt.PartnerMgr.noMsgStorage is set to false and thus the message body is stored to the file system, and
the ratio of the SAP BC 4.6 processing time per request to the SAP BC 4.7 processing time per request
reaches up to a factor of 4 for large documents. For the volume series scenarios, in which the parameter
watt.PartnerMgr.noMsgStorage is set to true, the performance increase is also especially high for large
documents, and the ratio of the SAP BC 4.6 processing time per request to the SAP BC 4.7 processing time
per request reaches up to a factor of 1.5. For the scalability scenarios the performance increase is especially
large for that scenario, in which the parameter watt.PartnerMgr.noMsgStorage is set to true and the
parameter watt.PartnerMgr.xtn.store.fastAsyncMode is set to false, and the ratio of the SAP BC 4.6
processing time per request to the SAP BC 4.7 processing time per request reaches up to a factor of 2.9.
For the XSLT scenario the test results of SAP BC 4.7 show also a performance increase compared with the
processing time per request reaches up to a factor of 2 for large documents.
For the IDoc scenarios the test results of SAP BC 4.7 show only a slight performance increase compared
with the measurements of SAP BC 4.6 for the volume series, and the ratio of the SAP BC 4.6 processing
time per request to the SAP BC 4.7 processing time per request reaches up to a factor of 1.08. For the IDoc
scalabilty scenarios the performance increase compared with the measurements of SAP BC 4.6 is higher,
and the ratio of the SAP BC 4.6 processing time per request to the SAP BC 4.7 processing time per request
reaches up to a factor of 1.3.
For the EDI scenario the test results of SAP BC 4.7 show also a performance increase compared with the
processing time per request reaches up to a factor of 1.8 for large documents.



5 Copyright

• No part of this publication may be reproduced or transmitted in any form or for any purpose without
the express permission of SAP AG. The information contained herein may be changed without prior
notice.
• Some software products marketed by SAP AG and its distributors contain proprietary software
components of other software vendors.
• Microsoft®, WINDOWS®, NT®, EXCEL®, Word®, PowerPoint® and SQL Server® are registered
trademarks of Microsoft Corporation.
• IBM®, DB2®, DB2 Universal Database, OS/2®, Parallel Sysplex®, MVS/ESA, AIX®, S/390®,
AS/400®, OS/390®, OS/400®, iSeries, pSeries, xSeries, zSeries, z/OS, AFP, Intelligent Miner,
WebSphere®, Netfinity®, Tivoli®, Informix and Informix® Dynamic ServerTM are trademarks of IBM
Corporation in USA and/or other countries.
• ORACLE® is a registered trademark of ORACLE Corporation.
• UNIX®, X/Open®, OSF/1®, and Motif® are registered trademarks of the Open Group.
• Citrix®, the Citrix logo, ICA®, Program Neighborhood®, MetaFrame®, WinFrame®, VideoFrame®,
MultiWin® and other Citrix product names referenced herein are trademarks of Citrix Systems, Inc.
• HTML, DHTML, XML, XHTML are trademarks or registered trademarks of W3C®, World Wide Web
Consortium, Massachusetts Institute of Technology.
• JAVA® is a registered trademark of Sun Microsystems, Inc.
• JAVASCRIPT® is a registered trademark of Sun Microsystems, Inc., used under license for
technology invented and implemented by Netscape.
• MarketSet and Enterprise Buyer are jointly owned trademarks of SAP AG and Commerce One.
• SAP, SAP Logo, R/2, R/3, mySAP, mySAP.com and other SAP products and services mentioned
herein as well as their respective logos are trademarks or registered trademarks of SAP AG in
Germany and in several other countries all over the world. All other product and service names
mentioned are trademarks of their respective companies.


Sap bc performance test

More Related Content

What's hot (20)

Viewers also liked (12)

Similar to Sap bc performance test (20)

More from ricardopabloasensio (20)

Recently uploaded (20)

Sap bc performance test