A Developer's Tale

If you have ever worked with the integration server (part of webMethods Product Suite) you have probably come in contact with flow. This is what this article is about.

Sometimes you´re in need of a way to limit the concurrent work for a specific service or subsystem. Basically, there are two ways to achieve this: 

1) Implementing a custom queue in java using e.g. a hash table, which hold the information about current executions (threads)
2) Using publish/subscribe and the built in trigger functions for controlling concurrent executing threads and queue size.

In order to decide which way to go I would first decide how important the response time of you particular service or flow is. If you are running some really time critical real time service (e.g. web service for transfering money or some other important stuff) were you must keep the response times to a minimum, I would probably go for the hash table. But if it´s not that critical and you have some tens of a seconds to spare, and it´s ok with an occasional delay of a few seconds every now and then – I would in 9 times out of ten go for the publish/subscribe solution.

This article describes the latter one – using the built in publish/response mechanism.

Local or Broker

First, decide if you should use local publishing (not outside the IS) or publish to the broker. If you can keep the work to a single IS, you should use local publishing (at least that´s what I prefer).

Setting our limits

To control the number of current active threads executing a service (configured i the trigger) you first have to decide a limit appropriate for you system. Lets decide that 10 concurrent threads are enough for us. Begin by setting the concurrent threads to 10 on your trigger, continue with max queue size and set this to 10 as well. The queue shows queued documents + the current executing threads, which means if the concurrent thread limit is equal to the queue size, the curreny queue size (persisted documents, we’ll come to this later) is allways equal to the number of executing threads.

Now, you might think were done – cause it sure seems so – but no!
The Integration Server actually ignores the maximum queue size, making it add more documents to the queue even though the configured limit is hit. To avoid this we have two choices.

Controlling the concurrent work

The first (and ugly one) is to set the watt.server.publish.local.rejectOOS property to true in extended settings.
This makes all local publish’s fail with the status message “Capacity exceeded” if the queue is full. After a publish you can check for this status message to see of the publish were successfull or not. Since this affect all other services doing local publishing you have to be very carefull you´re not breaking anything. Another dawback with this approach is that it generates an error in the error log, which may or may not be usefull depending on your solution. To avoid this (and the local publish issue), consider the second approach:

The other option, also the one I usually prefer is not to publish at all if the queue is full. There are some services in the WmRoot package which you can use to retrieve alot of information about a specific trigger. To see the WmRoot package in the Developer, set the watt.server.ns.hideWmRoot to false. To get the current processing status of a specific trigger, call wm.server.triggers:getProcessingStatus. This gives you (among other things) the Persisted Queue Count, which is the number of currently executing threads + any other documents in queue. If this magic little number is equal to or exceeds (god forbidden) your configured limit, you should not publish the document at all.

What now?

So, what are we gonna do with the request we could not complete? Since the system probably is overloaded, you either skip this particular step if it´s possible and continue – or you gracefully return an error code with a message saying the particular system is unavailable.

Summary

This is just one of many many ways to limit the concurrent work withing the integration server. It may or may not work (or fit) for your particular system (or implementation). However, it’s really easy to implement, it does not require alot of code (you can do it 100% in flow) and it´s easy to configure (keep you limit configured in a TPA or some other configuration file). So, think twice before ruling out this solution as an option – it might be exactly what you need :)

I will add a sample packege for download as soon as I have some spare time to put one together.

If you are using BlogEngine.Net on a site hosted by Surftown you have probably noticed that you get prompted with a credentials popup every now and then. This is because Surftown are running their applications in Medium Trust, which means you (NETWORK_SERVICE to be exact) don´t have any write access to the App_Data folder. Turn off “custom errors” on web.config and you´ll see a permission denied error.

Since the application is running under the NETWORK_SERVICE user we must give that user write privileges to the App_Data folder and all subfolders and files. Luckaly, surftown are nice enough to allow this and has also provided a script you can run to give the NETWORK_SERVICE user write access to any folder you wish. However, for security reasons you should keep the write access privileges to a minimum!

You can find the script here.
For all of you that don´t speak swedish – here comes a translation.

1. Create an aspx file and name it to change.aspx
2. Add the code below
3. Upload the file to a domain other than the domain where you keep your blog, e.g. create a temporary sub domain
4. Verify the ASP.NET version is set to 2.0 on the surftown control panel
5. Surf to the change.aspx page
6. You will be prompted by a credentials box. Supply your ftp credentials.
7. Done! Hopefully it worked and your application should run smoothly from now.

The Script

Download it here

   1:  <%@ Page Language="C#" %>

   2:   

   3:  <%@ Import Namespace="System.Security.AccessControl" %>

   4:  <%@ Import Namespace="System.Security.Principal" %>

   5:  <%@ Import Namespace="System.IO" %>

   6:   

   7:  <script runat="server">

   8:          protected void ChangeBT_Click(object sender, EventArgs e)

   9:          {

  10:                  // Get path from TextBox

  11:                  string path = PathTB.Text;

  12:                  

  13:                  // The account which should be granted access

  14:                  NTAccount acc = new NTAccount(@"NT AUTHORITY\NETWORK SERVICE");

  15:   

  16:                  // Rights which should be changed

  17:                  FileSystemRights rightName = FileSystemRights.Modify;

  18:                  AccessControlType right = AccessControlType.Allow;

  19:   

  20:                  // Inherit to all subdirectories and files

  21:                  InheritanceFlags iFlags = InheritanceFlags.ContainerInherit | InheritanceFlags.ObjectInherit;

  22:                  PropagationFlags pFlags = PropagationFlags.InheritOnly;

  23:   

  24:                  // Get current filesecurity object

  25:                  DirectorySecurity security = Directory.GetAccessControl(path);

  26:   

  27:                  // Create new rule

  28:                  FileSystemAccessRule rule = new FileSystemAccessRule(acc, rightName, iFlags, pFlags, right);

  29:   

  30:                  // Add new rule to the security object

  31:                  security.AddAccessRule(rule);

  32:   

  33:                  // Update file access control

  34:                  Directory.SetAccessControl(path, security);

  35:   

  36:                  UpdatedLB.Visible = true;

  37:          }

  38:  </script>

  39:   

  40:  <html>

  41:  <body>

  42:          <form id="Form1" runat="server">

  43:          <p>

  44:                  Path: (d:\hshome\username\yourdomain\foldername)<br />

  45:                  <asp:TextBox ID="PathTB" runat="server" Width="250px" />

  46:          </p>

  47:          <p>

  48:                  <asp:Button ID="ChangeBT" runat="server" Text="Change Permissions" onclick="ChangeBT_Click" />

  49:          </p>

  50:          <p>

  51:                  <asp:Label ID="UpdatedLB" runat="server" Text="Permissions changed!" Visible="False" />

  52:          </p>

  53:          </form>

  54:  </body>

  55:  </html>

The first thing I wanted for my BlogEngine powered site was to write a widget.
Since I haven´t dived very deep into the BE.NET code I´ll start with a simple widget that displays some profile information and a small image, like an avatar.
Basically, a classic "About Me" widget that will replace the TextBox widget that comes along with the source.

The basics

A widget consists of two elements; a widget control (ordinary user control) containing the UI and logic, and an edit control which both must inherit from WidgetBase and WidgetEditBase. They has to be placed in the widgets folder in the BlogEngine.Net project for it to work. BE.NET then uses reflection to dynamically load new widgets upon compile time. Every widget also has it´s own "storage" for storing widget-related data. Just call WidgetBase.GetSettings() to retrieve a StringDictionary and WidgetEditBase.SaveSettings(StringDictionary d) to save. The widget framework will take care of all the serialization and deserialization for you.

Step 1

Start with creating a new folder - lets call it AboutMe - in the widgets folder.


Then add 2 usercontrols; widget & edit and make them inherit from WidgetBase and WidgetEditBase

Now we have to override some properties/functions in the widget control:

The Name property decides the name for the property and it _must_ be the same as the folder name.
The IsEditable property tells the widget engine whether or not the widget is editable.
The LoadWidget() function is a widgets equivalent to Page_Load.

If you compile it now you should be able to add your new widget to the sidebar.
Dont forget to add the WidgetZone in your master page.

Yet another blog has come to life. Blood, sweat, tears and thousands of man hours was required to give birth to this magnificent blog... just kidding,  downloaded BlogEngine.Net, a nice theme from NodeThirtyTree, compiled and was good to go in a few hours; Sweet 

Anyway - for quite some time I´ve wanted to start a blog. Not the classic "write-about-myself-blog", but a blog about pretty much anything that are of interest for me. As I am a developer, most of the posts are probably gonna be about some technical topic - as it´s my absolute favourite topic. It´s intended solely for myself as an archive of technical information, achievements, howto's, how I solved certain problems and much more. However, if someone find a particular post interesting, helpfull... or just plain awefull - feel free to comment 

About me

So, who am I then?
Simply put - a 30+ nerd with love for programming, sports and fantasy books... and any book by Ken Follet.
When I´m not learning some new technology (MVC at the moment), reading a good book or out running, I´m probably enjoying a nice single malt scotch - smoky as hell! 
Well, thats a little bit about me - happy reading!

//Mike>