Loop/Test/Loadtesting

From MozillaWiki
< Loop‎ | Test
Jump to: navigation, search

Overview

Summary

Loadtesting Loop is currently performed with loadsv1 tool. Loadsv2 is under development (stay tuned for more info...)

Loadtesting should be performed against

  • loop-server
  • msisdn-gateway

It's not necessary to loadtest against loop-client

Guidelines for loadtesting with loadsv1 tool are provided below.


Loadsv1

Installing Loop-Server and the Loads tool (on Localhost or AWS)

  • refer to github README


Installation

loop-server

$ git clone https://github.com/mozilla-services/loop-server
$ cd loop-server/loadtests </pred>
$ make install

msisdn-gateway

$ git clone https://github.com/mozilla-services/loop-server
$ cd msisdn-gateway/loadtests </pred>
$ make install


Pre-requisite

  1. edit loadtest config file
  2. enter desired values for current test
$ vim /loadtests/config/megabench.ini


Run Test There are 3 commands available for testing with loadsv1:


#toc


MSISDN - loadtesting

There are 2 staging environments for msisdn-gateway

  • msisdn.stage.mozaws.net - for end-2-end testing (NOT for loadtesting!)
  • msisdn-loadtest.stage.mozaws.net - for loadtesting

see:

configuration file on the Stage server: /data/msisdn-gateway/config/production.json

#toc


Test Execution

Summary

Of the three tests available, make megabench will probably be the one you use most. make test is largely for a quick smoke test and make bench doesn't do anything you can't do with the other two.

For information about test monitoring:

Loop/Test/Monitoring


#toc


make megabench

In Terminal $ make megabench

./venv/bin/loads-runner --config=./config/megabench.ini --user-id=bonzogonzo --server-url=https://loop.stage.mozaws.net:443 loadtest.TestLoop.test_all
[2015-05-21 16:00:21,868][11285] Detached. run --attach to reattach


'Open the following URL in browser:' https://loads.mozilla.services.com

  • Login with credentials provided by Loop dev/QA team
  • You will see real-time output from your test run in your browser

#toc


make test

Description

Running $ make test allows you to kick off a loadtest locally against a remote endpoint, but then receive test output locally.

Optional: if you want to specify a test endpoint other than the default (see: Makefile), you can do it like so:

$ make test SERVER_URL=https://loop.stage.mozaws.net
 - or -
$ make test SERVER_URL=https://msisdn.stage.mozaws.net

WARNING: Do not launch loadtests against any PRODUCTION server, ever!


Example Output

$ make test SERVER_URL=https://loop.stage.mozaws.net
SIMPLE_PUSH_URL=https://call.stage.mozaws.net/ ./venv/bin/loads-runner --config=./config/test.ini --server-url=https://loop.stage.mozaws.net loadtest.TestLoop.test_http_calls
[                                                                                                                                                                                                                                 ]   0%/Users/rpappalardo/git/services-test/build/loop-server/loadtests/venv/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
[=================================================================================================================================================================================================================================] 100%
Duration: 4.99 seconds
Hits: 17
Started: 2015-05-21 22:26:10.930852
Approximate Average RPS: 3
Average request time: 0.28s
Opened web sockets: 0
Bytes received via web sockets : 0

Success: 1
Errors: 0
Failures: 0


Slowest URL: https://loop.stage.mozaws.net/     Average Request Time: 1.121348

Stats by URLs:
- https://loop.stage.mozaws.net/                        Average request time: 1.121348  Hits success rate: 1.0
- https://loop.stage.mozaws.net/calls/Dko0_Uvdks8       Average request time: 0.689918  Hits success rate: 1.0
- https://loop.stage.mozaws.net/rooms                   Average request time: 0.429905  Hits success rate: 1.0
- https://loop.stage.mozaws.net/rooms/C4lqDXc6W3M       Average request time: 0.189937375       Hits success rate: 1.0
- https://loop.stage.mozaws.net/registration            Average request time: 0.16524675        Hits success rate: 1.0
- https://loop.stage.mozaws.net/call-url                Average request time: 0.163699  Hits success rate: 1.0
- https://loop.stage.mozaws.net/calls?version=200       Average request time: 0.163517  Hits success rate: 1.0

Custom metrics:
- delete-room : 1
- register : 4
- generate-call-url : 1
- refresh-room-presence : 1
- initiate-call : 1
- leave-room : 3
- room-with-context : 1
- join-room : 3
- create-room : 1
- list-pending-calls : 1
- num-participants-3 : 1
SIMPLE_PUSH_URL=https://call.stage.mozaws.net/ ./venv/bin/loads-runner --config=./config/test.ini --server-url=https://loop.stage.mozaws.net loadtest.TestLoop.test_websockets
[                                                                                                                                                                                                                                 ]   0%/Users/rpappalardo/git/services-test/build/loop-server/loadtests/venv/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
[                                                                                                                                                                                                                                 ]   0%e
[=================================================================================================================================================================================================================================] 100%
Duration: 81.80 seconds
Hits: 17
Started: 2015-05-21 22:26:16.133047
Approximate Average RPS: 0
Average request time: 0.24s
Opened web sockets: 7
Bytes received via web sockets : 1350

Success: 1
Errors: 0
Failures: 0


Slowest URL: https://loop.stage.mozaws.net/     Average Request Time: 0.713177

Stats by URLs:
- https://loop.stage.mozaws.net/                        Average request time: 0.713177  Hits success rate: 1.0
- https://loop.stage.mozaws.net/calls/KR414mzrf-s       Average request time: 0.359527  Hits success rate: 1.0
- https://loop.stage.mozaws.net/calls/bYhXDs86FTQ       Average request time: 0.358428  Hits success rate: 1.0
- https://loop.stage.mozaws.net/calls/vqp9-BYyYBQ       Average request time: 0.357578  Hits success rate: 1.0
- https://loop.stage.mozaws.net/calls/Nbi4t9_pLAc       Average request time: 0.356943  Hits success rate: 1.0
- https://loop.stage.mozaws.net/registration            Average request time: 0.1638505 Hits success rate: 1.0
- https://loop.stage.mozaws.net/call-url                Average request time: 0.16342975        Hits success rate: 1.0
- https://loop.stage.mozaws.net/calls?version=200       Average request time: 0.16315475        Hits success rate: 1.0

Custom metrics:
- websocket-basic-callee-messages : 5
- websocket-supervisory-caller-messages : 2
- list-pending-calls : 4
- generate-call-url : 4
- initiate-call : 4
- websocket-ringing-callee-messages : 3
- websocket-basic-caller-messages : 5
- websocket-ringing-caller-messages : 3
- register : 4


#toc


Loadtest - Tweaking

loadtests can be configured in the code...


#toc


Reference

Monitoring


Ops / Infra


Loads Tool and Cluster


#toc