Cloud Foundry Performance Testing

Have been wondering for sometime how we can measure the overall response of a known Cloud Foundry PaaS target. What kind of metrics can we use ?

As a first attempt, let’s review the PAT tool as reviewed by Dr. Nic. Look here: https://blog.starkandwayne.com/2014/08/02/ask-pat-how-long-does-it-take-to-push-apps/

and if you are really keen, you can hack your own version of PAT from the Github repo here: https://github.com/cloudfoundry-incubator/pat

My ideas involved a test harness to hit several/many CF targets simultaneously and let it all run for a few days. This would give a larger overview across a wider time span. We could see the highs and lows using some metrics – which times of day are bad/good – which days of the week are good ?  Do background tasks on the target CF influence or results ? Are they doing backups while we push our app to them ?

Maybe another test harness to bang our app on each CF target every n seconds/minutes/days ?

Then we could publish those metrics so we all have a better of of which PaaS is rubbish like CloudBees and which ones are sterling like IBM BlueMix and Anynines.com 🙂

 

 

Gradle Properties in Cloud Foundry Closure

See notes here: https://github.com/cloudfoundry/cf-java-client/blob/master/cloudfoundry-gradle-plugin/README.md

1. according to gradle documentation, we can put our PaaS cloud foundry target credentials into

~/.gradle/gradle.properties

like this:

cf.username=’men@work.com’

with single quotes. If we do that, we get this result:

:template-project:cf-login
 Authenticating to 'https://api.run.pivotal.io' with username ''men@work.com''
 POST request for "https://login.run.pivotal.io/oauth/token" resulted in 401 (Unauthorized); invoking error handler
 :template-project:cf-login FAILED

FAILURE: Build failed with an exception.

but removing the single quotes makes it work.

NOTE: Gripe !! the credentials in gradle.properties take precedence over those declared in our gradle script closure !!! So if we deploy the same app to several different PaaS targets we can not declare credentials in gradle.properties we just need a build.gradle script for each PaaS target  😛

2. plugin cf-push where there is already an existing service leaves orphan services that were originally attached. Is this because of the “rediscloud-${randomWord}” random word declaration in the cloud foundry closure ?
3. same as 2. but also looses the uri as there is no/or a faulty rebind of uri to our app

 

4. cf-push of new app that does not already exist appears to work, correctly builds services and binds the uri correctly.

 

5. also note that ~/.gradle/gradle.properties needs a password too, so:

cf.password=porkypig

For Our Beautiful Friends@Anynines.com

julian.fischer@anynines.com
jweber@anynines.com
fdrees@anynines.com

Hi Guys and Dolls !

Just a note to say a GR8 BIG Thank You to each and everyone of you and all your clever crew too – thanx for all your super efforts on the Anynines PaaS project! It’s such a pleasure to work with a solid reliable platform like Anynines.com

I have been surprised that Anynines is not more widely known across the vast internet. To improve that situation, wanted to make a nice introductory tutorial. It could help new users ease their way onto Anynines.

With the kind help of Floor, who provided html color codes and font choices used on Anynines, it was possible to create something that looks (almost) like a native Anynines page.

It’s been some time in the making and it was so much fun too ! Am not worried if you all think it is useless or unnecessary – we can bin it – no worries. Alternatively if you think it might be useful or if you have any ideas or suggestions, or we need to add other topics, then send me some ideas. I’ll put it up on my github acct when it’s finished, so you can clone from there into your own account.

You can always rename it, but as i like george harrison’s album, i’ve nicknamed our tut ‘cloud9‘ 🙂

So enjoy: http://cloud9.de.a9sapp.eu

Image

( The zip file example is not available yet as i dont have enough resource for that )

Again, thanx for the pleasure !

kind regards

jim

PS – yes i know this tut was for CF v5 and we’ll need to update to CF v6 samples – you were too quick!!! :-}

Gradle Plugin for Cloud Foundry for IBM Blue Mix

IBM came late to the PaaS party. Perhaps it was a tactical choice to avoid Cloud Foundry version one. Their delay allowed the cloud foundry platform to evolve so now, with IBM Blue Mix, we have a quick performing version two of cloud foundry.

To use version one of C/F we had to use command line tools like VMC and CF v5.4.7. For V2, we need more modern command line tools like the recently introduced CF ver. 6 which is written in the GO programming language. It avoids the need for a Ruby programming installation as the GO version of CF carries all it’s own logic as a single bundle. Hurrah ! I use CF v6.0.2 to talk to Blue Mix.

But as i am still looking for better ways to deploy apps to cloud foundry platforms, i like to use a build tool and, as i’ve reported in prior posts, the gradle build tool is my current flavor of the month. With it, we can get plugin components that do much of the work and add new pieces of functionality.

So for my gradle build scripts, i’ve picked up on the cloud foundry plugin for gradle. Scott Frederick explains it in more detail here if you have a need to know.

Back to IBM’s Blue Mix. Here is part of my gradle build script to deploy an app to IBM’s Blue Mix platform. We need to declare the plugin we need and the location where it can be downloaded from, so :

 

import org.apache.ivy.plugins.resolver.URLResolver
apply plugin: 'cloudfoundry'
buildscript {
 repositories {
 mavenCentral()
 maven { url "https://oss.sonatype.org/content/repositories/springsource-milestones" }
 }
 dependencies {
 classpath group: 'org.cloudfoundry', name: 'cf-gradle-plugin', version: '1.0.2'
 }
}

Notice that i’ve picked v1.0.2 as this version WILL talk to later cloud foundry v2 platforms. To reach  the earlier version one platforms, we use v1.0.0 of this plugin.

Next we need to declare some values inside the closure used by this plugin.  In particular, the target API address needs to be https://api.ng.bluemix.net, for your username and password values, put in the ones you used to join IBM’s beta plan for Blue Mix. Or do so here.

Then pick the name your want your app to be. Fill that name into the application field below and repeat it in the URI in place of my caelyftemplate2 choice. The full closure looks like this :

cloudfoundry {
 target='https://api.ng.bluemix.net'
 space = 'dev'
 username = 'men@work.com'
 password = '******'
 file = new File('build/libs/template-project-1.2.0.war')
 uri = 'http://caelyftemplate2.ng.bluemix.net'
 application = 'caelyftemplate2'
 }

and that should do it. I’m deploying a war file from within my gradle build folder. You might need to change the new File(‘xxx’) choice to point to your own war file.

Wrap the above text into a build.gradle text file. You need an installed gradle build tool from http://www.gradle.org/

To see what cloud foundry commands we have from the plugin, we can do a ‘tasks’ request to show them. Here is a sample output on my system from such a command:

jim@a1128-uk:/media/jim/project$ bash gradlew -b gradle.build tasks
 Parallel execution is an incubating feature.
 :tasks
------------------------------------------------------------
 All tasks runnable from root project
 ------------------------------------------------------------
Build Setup tasks
 -----------------
 init - Initializes a new Gradle build. [incubating]
 wrapper - Generates Gradle wrapper files. [incubating]
Cloud Foundry tasks
 -------------------
 cf-app - Displays information about the application deployment
 cf-apps - Lists applications running on the targeted Cloud Foundry platform
 cf-bind - Binds services to an application
 cf-create-service - Creates a service, optionally binding it to an application
 cf-delete - Deletes an application deployment
 cf-delete-service - Deletes a service
 cf-deploy - Deploys a variants of an application
 cf-env - List application environment variables
 cf-login - Logs in to the target Cloud Foundry platform
 cf-logout - Logs out of the target Cloud Foundry platform
 cf-logs - Shows the contents of log files
 cf-map - Maps a URI to an application
 cf-push - Pushes an application
 cf-restart - Restarts an application
 cf-scale - Scales application instances up or down
 cf-service-plans - Displays information about service offerings
 cf-services - Displays information about service instances
 cf-set-env - Sets environment variables to an application
 cf-spaces - Displays available spaces
 cf-start - Starts an application
 cf-stop - Stops an application
 cf-swap-deployed - Swaps the URIs for deployed variants of an application
 cf-target - Displays information about the target Cloud Foundry platform
 cf-unbind - Unbinds services from an application
 cf-undeploy - Undeploys variants of an application
 cf-unmap - Unmaps a URI from application
 cf-unset-env - Deletes environment variables from an application
Help tasks
 ----------
 dependencies - Displays all dependencies declar
ed in root project 'template-project'.
 dependencyInsight - Displays the insight into a specific dependency in root project 'template-project'.
 help - Displays a help message
 projects - Displays the sub-projects of root project 'template-project'.
 properties - Displays the properties of root project 'template-project'.
 tasks - Displays the tasks runnable from root project 'template-project'.
To see all tasks and more detail, run with --all.

BUILD SUCCESSFUL

Total time: 31.991 secs

You can see one of the deployment commands. The CF-PUSH task asks gradle to deploy an app to BlueMix.

First, you need to login before running the CF-PUSH command, so do CF-LOGIN which uses your credentials from within the closure.

Make sure you change directories into the folder with the above build.gradle script. Then from a command line, try this:

gradle cf-login cf-push

This command asks the gradle build tool to issue a login to the BlueMix platform at the defined target using the credentials from within the closure. As i’m using Lubuntu Linux 14.04, i need to call the ‘bash’ tool  my command looks like this (apple/windoze people won’t need bash):

jim@a1128-uk:/media/jim/project$ bash gradle -b build.gradle cf-login cf-push
Parallel execution is an incubating feature.
:cf-login
Authenticating to 'https://api.ng.bluemix.net' with username 'men@work.com'
Authentication successful
:cf-push
Updating application caelyftemplate2
Uploading build/libs/template-project-1.2.0.war
Starting caelyftemplate2
-----> Downloaded app package (34M)
-----> Downloaded app buildpack cache (4.0K)
Buildpack Version: 20140425-1807
-----> Downloading IBM 1.7.0 JRE from http://file.icap.cdl.ibm.com/icapShared/jre/ibm-java-jre-7.0-6.0-linux-x86_64-small-footprint-uncompressed-jar-20140116.tar.gz (0.0s)
 Expanding JRE to .java (1.1s)
Downloading from output/wlp/com.ibm.ws.liberty-2014.3.0.0-201404251807.tar.gz ... (0.0s).
Installing archive ... (0.6s).

-----> Uploading droplet (126M)

Checking status of caelyftemplate2
 1 of 1 instances running (1 running)
Application caelyftemplate2 is available at http://http://caelyftemplate2.ng.bluemix.net

BUILD SUCCESSFUL

Total time: 2 mins 58.293 secs

That’s enough for the moment. If i get a mo. i’ll post a small gradle project that has all this in it. Ok now, go forth and hack !  🙂

 

 

 

 

 

Cloud Foundry Improves Java Support

Until now, users running Java applications requiring an application server typically relied on Apache Tomcat as the default container in the Cloud Foundry Java Buildpack.

Tomcat is a great light-weight option for many Java web applications supporting the Servlet specification, but there are also many applications that use some Java EE features such as

  • EAR – enterprise archive packaging
  • JMS – java messaging service
  • JDBC Data Sources
  • Cloud Foundry’s Loggregator – a unified log stream
  • Blue-green deployment – a release technique to reduce downtime and risk by running two identical production env.s called Blue and Green
    ( CF version 6 only )
  • plus other application server-specific features not available on Tomcat

The new Oracle WebLogic buildpack along with the recently announced JBoss buildpack, now enables new types of Java applications to run natively on Cloud Foundry.

More Java News on Cloud Foundry

http://blog.gopivotal.com/cloud-foundry-pivotal/products/cloud-foundry-expands-java-support-with-new-oracle-weblogic-buildpack

Build Pack Debugging

Do you need to debug build packs ? No, neither do i, but when i grow up and become a hacker, i’m gonna read this: https://github.com/cloudfoundry/java-buildpack/blob/master/docs/debugging-the-buildpack.md

Install Cloud Foundry CF Command Line Tool – Part One

Microsoft Windows XP
Microsoft Windows XP
HP Pavillion
HP Pavillion

Ok, please don’t laugh, but i still use windowsXP as it’s rather reliable and un-complicated. Have an aging HP Pavillion desktop computer that i’ve had to bring out of storage. Was in a rather cold place so this may caused some of my troubles. After power-up this system would re-boot itself every few hours, then it became so bad it would reboot itself every few minutes and disk access was  problematic too. Think it finally trashed my hard disk as it would not read that either.

Had a copy of Lubuntu 13.10 install disk that i put into this machine. Did you know it has a memory tester utility on the boot loader screen ? Well i never noticed that until now. So ran the memory utility tester. It ran for 30 minutes and reported a bunch of memory failures. Could not tell which bank of memory it was, as my system has four banks of 512MB. And this system needs a minimum of 2 banks, so i took out the left-most two chips and ran the memory test again. Still a few memory failures, so took out one of the installed banks and replaced it with one i had taken out, not knowing exactly which chips had the problems.

Ubuntu Live CD plus 2 broken memory chips
Ubuntu Live CD plus 2 broken memory chips

Restarted system and ran memory tests again for about 30 minutes and – bingo ! – no memory failures. OK that leaves me 1GB of memory but that’s enough for XP ! Wonderful !

Ok, next problem was a blitzed XP operating system, had to re-install that. Using another utility from the ubuntu dvd, i could see that when HP shipped me this box,  HP had kindly created a second disk partition holding a complete restore pack. By pressing keyboard F10 function key during reboot, the restore utility kicked in. It promised NOT to blitz any of my data, so i let it finish, and after about 30 minutes, i had a fresh XP system with my data – what a relief. The wireless network came up almost at once, after putting in the wireless WPA2 security key.

The next step was to restore all the lost user programs i had installed before. Yes, my Ruby, RubyGems and CF cloud foundry command line tools were lost. How to put those in again ?

Part Two reveals all !

 

 

Using gradle cloud foundry plugin to target Anynines non-Pivotal address

FYI Using gradle cloud foundry plugin to target non-Pivotal address

Ref.: https://github.com/cloudfoundry/cf-java-client/tree/master/cloudfoundry-gradle-plugin

You may or may not be interested, but FYI, i’ve been able to use your gradle plugin to push our Caelyf Sample Template to a different cloud foundry platform other than Pivotal. This automates the task so we do not need to use the CF tool. Details are below but salient points about your plugin are:

1. The user credentials from the ~/.gradle/gradle.properties take precedence over those declared within the gradle closure. Since i have two different set of credentials one for Pivotal and another to the AnyNines service, my gradle.properties received an http 401 – not authorised. Had to change file name to .bak so that the plugin would be forced to use credentials from closure. Then inserted my username and password into the gradle script. Could we make the plugin use script credentials before trying credentials from gradle.properties ???

2. Anynines target address is different, so changed that.

3. For this test, did NOT use any persistence services like redis, etc. They are optional for our template.

4. Used gradle default task declaration to make it all happen ‘by magic’ -> defaultTasks ‘cf-login’,’cf-push’

5. I had commented out the target URI address when doing the Pivotal upload, so the first cf-push to A9S got our app up to the target ok, but anynines had NO external URL address to bind the app to. I COULD see it using the CF command but with no URL. So changed the uri declaration to give our app an external URL.
From my terminal using script below :
$ gradlew -b builda9s.gradle –info

and bingo, Houston, we have lift-off !!

Our successfully deployed PaaS service can be seen here, thanx to you all and to the clever people at Anynines.com :
http://caelyf.de.a9sapp.eu/

The European platform for the Anynines PaaS service can be found here:
http://www.anynines.com/

When you have an account, you can use this address to see their console. It does not have the featureset as the http://www.gopivotal.com/paas console does. See here:
https://customerpanel.de.a9sapp.eu/

// ----------------------------------------
// caelyftemplate builda9s.gradle - 
// used to push to anynines target using gradle plugin
// see: https://customerpanel.de.a9sapp.eu/
import org.apache.ivy.plugins.resolver.URLResolver
apply plugin: 'groovy'
apply plugin: 'cloudfoundry'

defaultTasks 'cf-login','cf-push'

buildscript {
 repositories {
 mavenCentral()
 maven { url "https://oss.sonatype.org/content/repositories/springsource-milestones" }
 }
 dependencies {
 classpath group: 'org.cloudfoundry', name: 'cf-gradle-plugin', version: '1.0.1'
 }
}
repositories {
 mavenCentral()
}
cloudfoundry {
 target='https://api.de.a9s.eu' // cloud foundry V2 target for anynines
 space = 'development'
 username = 'jimmy'
 password = '****'
 file = new File('build/libs/template-project-1.1.2.war')
 uri = 'http://caelyftemplate.de.a9sapp.eu/'
 application = 'caelyftemplate'
 memory = 512
 instances = 1
} // end of closure

// ----------------------------------------------
// my console log:
Starting Build
Settings evaluated using empty settings script.
Projects loaded. Root project using build file 'C:\Software\Groovy\caelyf\template-project\builda9s.gradle'.
Included projects: [root project 'template-project']
Evaluating root project 'template-project' using build file 'C:\Software\Groovy\caelyf\template-project\builda9s.gradle'.
Compiling build file 'C:\Software\Groovy\caelyf\template-project\builda9s.gradle' using StatementExtractingScriptTransformer.
Compiling build file 'C:\Software\Groovy\caelyf\template-project\builda9s.gradle' using BuildScriptTransformer.
All projects evaluated.
No tasks specified. Using project default tasks 'cf-login', 'cf-push'
Selected primary tasks 'cf-login', 'cf-push'
Tasks to be executed: [task ':cf-login', task ':cf-push']
:cf-login (Thread[main,5,main]) started.
:cf-login
Executing task ':cf-login' (up-to-date check took 0.0 secs) due to:
 Task has not declared any outputs.
Authenticating to 'https://api.de.a9s.eu' with username 'jimmy'
Connecting to 'https://api.de.a9s.eu' with username 'jimmy'
Authentication successful
:cf-login (Thread[main,5,main]) completed. Took 9.546 secs.
:cf-push (Thread[main,5,main]) started.
:cf-push
Executing task ':cf-push' (up-to-date check took 0.0 secs) due to:
 Task has not declared any outputs.
Connecting to 'https://api.de.a9s.eu' with username 'james.northrop@orange.fr'
Creating application caelyftemplate
Uploading 'build\libs\template-project-1.1.2.war'
Starting caelyftemplate
-----> Downloaded app package (31M)
Downloading JDK...
Copying openjdk-1.7.0_25.tar.gz from the buildpack cache ...
Unpacking JDK to .jdk
Downloading Tomcat: apache-tomcat-7.0.41.tar.gz
Copying apache-tomcat-7.0.41.tar.gz from the buildpack cache ...
Unpacking Tomcat to .tomcat
Copying mysql-connector-java-5.1.12.jar from the buildpack cache ...
Copying postgresql-9.0-801.jdbc4.jar from the buildpack cache ...
-----> Uploading droplet (68M)
Checking status of caelyftemplate
 0 of 1 instances running (1 starting)
 0 of 1 instances running (1 starting)
 0 of 1 instances running (1 starting)
 0 of 1 instances running (1 starting)
 0 of 1 instances running (1 starting)
 1 of 1 instances running (1 running)
Application caelyftemplate is available at http://http://caelyftemplate.de.a9sapp.eu/
:cf-push (Thread[main,5,main]) completed. Took 38.563 secs.
BUILD SUCCESSFUL
Total time: 1 mins 2.297 secs
Stopped 0 compiler daemon(s).