Monday, August 23, 2010

Capturing Page Load Times - FiddlerScript to the Rescue

So I have a client who has locations all over the world and we’ve been doing some analysis of some performance issues with SharePoint. We believe that we narrowed down the issue to SQL server not having enough memory. The key indicator for SQL Server memory is SQL Server Buffer Manager: Page Life Expectancy (PLE) – anything less than 300 (seconds) isn’t going to perform optimally. However, we needed some “proof” that changing the SQL memory changed the performance – for people across the world. The solution, rather than buying and setting up expensive monitoring all over the world was to get some of the users to run Fiddler. If you’ve never seen the tool you should go check it out now. (You can go look at an article I wrote in 2006 if you want to see what I was thinking 4 years ago.)

The problem was that there’s no quick and simple way to get the session timers out of Fiddler. That is except that Eric Lawrence (the author of Fiddler) is a genius. He added scripting support to Fiddler so you can script the stuff you need. With a few looks at the cookbook, some trial and error, and a few pointers from Eric, I wrote some script that creates a tab separated value file which can be imported into Excel. The file has all of the key timers and the number of milliseconds that elapsed. You can go to Rules-Customize Rules (Ctrl-R) to customize the rules – or you can download the handy script editor. I added a tools menu option to write all of the session timers to a TSV and a context menu option to write the selected session timers to a TSV. The code is below if you want to do this yourself. By the way, I used a tab separated value file because I had a situation where some of my URLs had commas in them. (Don’t ask) I won’t guarantee this is the absolute best way to make this all work – but it’s functional.

public static ToolsAction("Write all session timers to TSV file")
function WriteAllSessionTimers()
{
       var fileName = GetSaveFileName();
       if (fileName != null)
       {
              var oSessions : Fiddler.Session[] = FiddlerApplication.UI.GetAllSessions();
              WriteSelectedSessionTimersToFileName(oSessions, fileName);
       }

}

public static ContextAction("Write selected session timers to TSV file")
function WriteSelectedSessionTimers(oSessions: Fiddler.Session[])
{
       var fileName = GetSaveFileName();
       if (fileName != null)
       {
              WriteSelectedSessionTimersToFileName(oSessions, fileName);
       }
}

public static function GetSaveFileName()
{
       var sfd : System.Windows.Forms.SaveFileDialog = new System.Windows.Forms.SaveFileDialog();

       sfd.Filter = "Tab Separated Value (*.tsv)|*.tsv|All files (*.*)|*.*";
       if (sfd.ShowDialog() == System.Windows.Forms.DialogResult.OK)
       {
              return(sfd.FileName);
       }
       else return (null);
}


public static function WriteSelectedSessionTimersToFileName(oSessions: Fiddler.Session[], filename: System.String)
{
       try
       {

              if (oSessions == null || oSessions.Length == 0)
              {
                     MessageBox.Show("Please select sessions first", "Warning");
                     return;
              }

              var sb: System.Text.StringBuilder = new System.Text.StringBuilder();
       sb.Append("Host\tPathAndQuery\tMimeType\tClientConnected\tClientDoneRequest\tServerConnected\tServerGotRequest\tServerBeginResponse\tServerDoneResponse\tClientBeginResponse\tClientDoneResponse\tTotalTimeMS\r\n");
              for (var looper=0; looper<oSessions.Length; looper++)
              {
                     var ts: System.TimeSpan = null;
                     var dtStart: DateTime = oSessions[looper].Timers.ClientConnected;
                     var dtEnd: DateTime = oSessions[looper].Timers.ClientDoneResponse;
                     ts = dtEnd - dtStart;
                     var mimeType = null;
                     if (oSessions[looper].oResponse.headers.Exists("Content-Type"))
                     {
                           mimeType = oSessions[looper].oResponse.headers["Content-Type"];
                           if (mimeType.indexOf(';') > -1) mimeType = mimeType.substr(0, mimeType.indexOf(';'));
                     }

                     sb.Append(oSessions[looper].host).Append('\t')
                           .Append(oSessions[looper].PathAndQuery).Append('\t')
                           .Append(mimeType).Append('\t')
                           .Append(oSessions[looper].Timers.ClientConnected.ToString("MM/dd/yyyy h:mm:ss.ffff ")).Append('\t')
                           .Append(oSessions[looper].Timers.ClientDoneRequest.ToString("MM/dd/yyyy h:mm:ss.ffff ")).Append('\t')
                           .Append(oSessions[looper].Timers.ServerConnected.ToString("MM/dd/yyyy h:mm:ss.ffff ")).Append('\t')
                           .Append(oSessions[looper].Timers.ServerGotRequest.ToString("MM/dd/yyyy h:mm:ss.ffff ")).Append('\t')
                           .Append(oSessions[looper].Timers.ServerBeginResponse.ToString("MM/dd/yyyy h:mm:ss.ffff ")).Append('\t')
                           .Append(oSessions[looper].Timers.ServerDoneResponse.ToString("MM/dd/yyyy h:mm:ss.ffff ")).Append('\t')
                           .Append(oSessions[looper].Timers.ClientBeginResponse.ToString("MM/dd/yyyy h:mm:ss.ffff ")).Append('\t')
                           .Append(oSessions[looper].Timers.ClientDoneResponse.ToString("MM/dd/yyyy h:mm:ss.ffff ")).Append('\t')
                           .Append(ts.TotalMilliseconds).Append("\r\n");
              }

              var sw : System.IO.StreamWriter = System.IO.File.CreateText(filename);
              sw.Write(sb.ToString());
              sw.Close();
              sw.Dispose();
       }
       catch (excpt: System.Exception)
       {
              MessageBox.Show(excpt.ToString());
       }
}

Categories: Professional | 0 Comments

Tuesday, August 10, 2010

Running Users Groups

A friend of mine, Chris Geier, posted a blog post titled "SharePoint: The User Group Phenomena" on the End User SharePoint site. He's pondering the whole users group question. Having been involved with users groups for just shy of 20 years, I feel like there are a few things that I should say about how users groups form, how they develop their own personality, and how they fail. I should say that I don't feel like I have the corner on the market here, I am just sharing what I know from my experience.

How Users Groups Form

A few weeks ago I got a message through my friends at Microsoft. A gentleman in town was looking for an users group where he could learn more about Team Foundation Server. He was using it but didn't feel like he was getting as much out of it as he could – and given the complexity of the product I can support that finding a users group is a good idea. Without getting into details the TFS group locally hasn't been very successful so there's no answer for him today. However, folks like myself, and some folks from the Developer Platform Evangelism group at Microsoft are encouraging him to start his own group. I've volunteered to reach out to my clients that use TFS and Microsoft has agreed to fund some food for an initial meeting. Will this turn into a new TFS group in Indianapolis? I don't know. However, this is one of the ways that users groups form. People are looking for a way to learn more about using a product and they try to gather together to share experiences. This is probably one of the best ways that users groups form. It is a pretty simple things.

Another group that I'm a part of is a rather informal meeting of the technical directors from various Christian churches around Indianapolis. It's candidly driven by one guy, Daryl Crype. Daryl is the technical director for Grace Community Church. Four years ago I reached out to him because I was being asked to support the production of our Sunday morning worship services at Hazel Dell Christian Church and I was completely out of my league. The entire depth of the group is a meeting roughly once a month at a restaurant where we each buy our own food and talk about the things that we're working on. It's an users group – but one with a lot less structure.

The SharePoint Users Group of Indiana which I lead (I've really got to fix that at some point) was born out of the passion of Bess Wuertz who was at the time it was launched working for the Indianapolis Airport. It was SharePoint 2003 in 2005 or 2006 (I don't remember) and the product was a lot less popular than it is now. Bess wanted to get folks together to share their experiences with the product. I stepped in after there were several meetings that Bess could no longer support. We agreed that I'd help bring some of my experience to ensure that the group continued to run. We've been running continuously since Bess and my discussion. We draw 30-50 people every other month. We've just introduced new leadership to the group and we're adding a second meeting for a slightly different audience of folks (end users and business users). We typically have a meeting every other month in the evening. Our end user/business user meeting will be done during the lunch hour on the intervening months.

Last year Darrin Bishop approached me about creating an users group for SharePoint in Illinois. Darrin's interest is in supporting the community and in creating a better group of people using the product so that folks can be successful. His group is growing and looks to be on the right track at the moment. For him the meetings are

And those are the good ways that users group form. They're born out of someone's need, a philanthropic desire to help the technical community, or out of a set of random conversations. I will say that there are a few "users groups" that are not exactly that. The other thing that can happen is that an individual or organization can decide that they need a marketing vehicle and that starting an users group is the answer. As a result the users group takes on a marketing slant. This clamps down on user interaction (after all if users help each other how will the consultant get work?) The bigger problem is that it tends to block out other consultants that might want to participate. (I believe that clients are like flowers and consultants are like bees – consultants cross pollinate ideas between clients. As a result they're an essential part of the ecosystem.)

I should say that not all users groups run by organizations are bad. It really depends on who's running them and whether they're trying to extract value out of the group itself or whether they're using the group to grow the potential opportunities in the market. It's a subtle but big difference.

Given that I run the SPIN users group and I run a consulting company I feel like I have to explain that I *NEVER* market my services from the front of the users group. As a point of fact, I don't even list my organization as a sponsor for the group. I put the same amount of money in as the other sponsors but I personally don't feel like it's "my" group. It's a community service. I do get people contact me because I lead the group to ask if I do consulting – I'll answer them but I am really conscious of the need to keep the group as a community group. To that end, I seek out ALL the consulting organizations doing SharePoint work in town and ask them for a small sponsorship rather than looking for a few high dollar sponsorships. I want everyone to have a small buy in for the groups long term success.

How Users Groups Run

I've seen many different ways of running. Evenings. Mornings. Lunch. Formal sponsors. Small per user fees. One presentation per meeting. Multiple presentations per meeting. Sponsors can present. Sponsors pay to present. While I have a few of my own thoughts I don't think these are the only answers. A few of the things that I've learned are:

Consistency is important – but not the only thing. Meeting in the same place at the same time on the same day of the month matters. Make a change and you'll see a drop in attendance. Depending on the change it can be 10% or so … but as much as 50%.
Communication is CRITICAL – If I want to control our meeting size down it's easy. Just don't communicate. Don't send the note out 3 weeks ahead of time. Don't send the reminder. I've used this technique when I've had problems with space to ensure that the people who were really interested could attend. Failing to send out a last minute reminder can be 20% of your audience.
Timing influences passion – If you do meetings at night you know that you're getting the folks who are truly passionate. If you do the events during the day you'll get folks who want to get out of work. I personally know a single mother who arranges for a baby sitter to be able to make some of our meetings because she's passionate about what she does and the content we deliver. It's easier for most folks – except consultants – to do meetings during the middle of the work day where they can block it off on their calendar.
Leave open space – One of the key things that Bess started with our group and a thing that I vigorously protect is the time that is designed for folks to talk – to socialize – and to get to know one another. One of the reasons we have a single presentation for an hour is to create extra time for folks to talk to each other. With two presentations or a longer presentation groups end up doing only presentations and no one talks.
Presentations provide focus – With a handful of exceptions, presentations are necessary to provide focus. Without them people show up and wonder what to talk about. We use presentations as our tool for getting folks together around a central topic. We give them a topic to discuss.

I'm sure there are tons of other things that we do that I don't even think of any more. If you are running a group and want to run an idea, thought, or problem by me feel free.

How Users Groups Fail

I've watched more than a few groups fail over the years, even one where I was officially holding the reigns. I can honestly say that groups ALWAYS fall apart from the leadership. Maybe they're not managing the political wars. Maybe all of the passion has gone out of the group. Maybe they've developed commitment cancer. Whatever the official cause of death for an users group it's always leaderships' fault.

I want to address three key problems because they're critically important to me.

Politics

There are cities where the business environment is so politically charged that I believe it's difficult to run a successful group. Let me pick on my neighbors in Chicago since I'm close enough to see what's happening and far enough away to not be able to fix it. The business market in Chicago is hyper competitive. Consulting companies always seem to be hungry and they always seem to be competing with one another for that one deal that will keep them afloat. The problem with this is that it makes it really hard for anyone (particularly sales and marketing folks) to come together for the common good of the market. As a result groups end up getting pulled in weird ways as the consulting companies protect their own interests. Chicago has what amounts to two SharePoint users groups because the respective consulting companies are controlling the group too tightly. Interestingly both are near failure. One of these days it's my hope that they'll figure out that there should be one group and everyone should focus on the development of the community and not on individual consulting company gains. (I guess I don't have to worry about speaking in Chicago for a while now.)

If leadership fails to put the users and the community first then there will be problems.

Commitment Cancer

In an users group you have a motivation problem. You can't motivate another person really. You don't have a financial incentive to offer them. You can't really publically shame them. You've got to rely on people's character to create the right results. One of the COMMON problems that I see with users groups is that they develop commitment cancer and it spreads. What is commitment cancer? Well, it's where someone on the steering committee or board makes a commitment and doesn't meet it. It is quietly pushed aside as people don't want to hurt anyone's feelings. Pretty soon there's another member who's not meeting their commitments – why should they? Where does it end? Most of the time, commitment cancer ends with the group shutting down. Why? Because nothing gets done. It's that network of commitments that keeps pushing the group forward. Without that commitment the group will wither and die. I've got one users group in Indianapolis that I'm no longer a part of because it has commitment cancer and the leadership is refusing to get treatment.

The Passion is gone

Another way an users group can fail is that the passion dies. This is a hard one. It's a hard one to see and it's a hard one to fix. Sometimes the market moves on. Sometimes the group was needed to fill a particular training need. Sometimes the leadership just isn't passionate about the spot in the community any longer. For instance, there was a time I was passionate about SQL server. I'm not really that passionate about it any longer. I feel like the information that needs to be out there is out there. While there's a vibrant SQL users group in Indy, it's not a place that I can spend my time. The one thing I can say about this is that everything has SOMEONE who's passionate about it.

In Parting…

I've seen people use users groups to help them find a job – good for them. However, it would be nice if they would repay what they got out of the group by coming back and supporting it.

Categories: Professional | 4 Comments

Monday, August 09, 2010

Conferences for the second half of 2010

I'm such a delinquent. I've got all of these great conferences that I'm doing the second half of the year and I've not had a chance to share what I'm up to.

First, I'll be at SharePoint Saturday Columbus next weekend. I'm doing a session titled "Solution Creation for the IT Pro without Semicolons" -- it's a lot of fun to put away Visual Studio and show folks what can be done without compiling anything. I spent a year where I was in Columbus every week for a few days – so it will be good to get back over and visit with folks I've not seen in a while.

August 24^th-27^th I'll be joining an absolutely stellar set of speakers at the Best Practices Conference SharePoint. I'm presenting a session titled "Making Development Design Decisions for SharePoint" and a session titled "Working Smartly with Workflow." The first session is some of the things that didn't fit into the SharePoint Guidance. Expect a raw look at the ways to look at making decisions on SharePoint. The second session will encompass more of the work from the SharePoint Guidance as well as work on the Microsoft Learning instructor lead course "Designing Applications for Microsoft SharePoint 2010" (10232). I'll also be sharing my love – and hatred – for SharePoint workflow. I can honestly say that I've done things that no one else has done with Workflow. Some of it I won't be doing again. Expect you'll know what's broken and what works when you walk out.

On October 20^th, I'll be delivering the first public session based on the SharePoint Shepherd's Guide for End Users 2010 at SPTechCon. In the session sub-titled "A day's walk in SharePoint with the Shepherd" should be a lot of fun as we walk through the key things that end users need to do – and some cool things that folks don't know that can be done. Later in the week I'll be doing a presentation "Protecting your SharePoint Farm from Evil Developers" which is particularly humorous since I do mostly development. I aim to strike a balance between IT Pro needs and Developer needs. The Friday's session is "Playing in the Sandbox: What an IT Admin Should Know" – We'll tear apart the idea of the sandbox, figure out what goes where, and how it impacts the farm. In this session we're focused on understanding how all of the pieces fit together. SPTechCon is a great place to go if you're in charge of a SharePoint deployment in your organization and you don't feel like you're really up to the challenge – you'll get what you need to be successful.

On November 1^st, I get the distinct pleasure to share the stage with Eric Shupps. We'll be delivering a SharePoint 2010 Professional Development Workshop at SharePoint Connections. This will be an action packed day of content as we try to cram the key things that you need to know to develop for SharePoint into a day. We're skipping right over the basics and dropping you in the land of hard decisions as we walk you through whether you should put your data in a standalone SQL database or in SharePoint – and what it means no matter what you decide. I don't know of another day session where you can grab two of the leading experts on SharePoint Development and drink from a fire hose of information.

If you haven't had an opportunity to go to a national SharePoint conference – this year is the year. It's some great content from the leading experts on SharePoint at all of the conferences.

Categories: Professional | 0 Comments

Friday, August 06, 2010

Book Review: The Time Paradox

The Time Paradox – The New Psychology of Time That Will Change Your Life may have a hyperbole for a sub-title but the insights that it provides are valuable. I try to look for new perspectives to view the world around me. Being a consultant and speaker means I meet lots of people and it helps to be able to understand where they're coming from. This book provided me another way to frame my thinking about how others think differently.

The short summary is that people have different ways that they view time. Some folks look to the past and view it negatively - -reflecting and remembering only those things that happened that were bad. Some look to the past and draw from it the warm moments like roasting marshmallows and making smores. Some folks are more focused on living for the moment, what the book calls a present hedonistic focus. Still others view the present from a fatalistic point of view. That is they believe everything is fated and that what they do doesn't matter. Finally, there are folks that focus on the future. They're willing to sacrifice today for a better tomorrow.

In reality there are aspects of each of these in every person. Sometimes we are able to live in the moment. Sometimes we're willing to remember those camping trips with our family. However, we all have tendencies towards one or more time perspectives. For instance, I have an interesting mix of future orientation and present orientation.

The book's web site allows you to calculate your perspectives on time using a standard scale. My future score is 4.15. (Pretty high) This isn't a surprise to the folks who know me. I replaced mulch beds with rock beds in places because it had a lower long term maintenance cost. I've replaced the siding on my house with concrete board that can't rot. My deck is made of composite materials that won't rot. My furnaces are both heat pumps with gas backup which is more expensive to install but has the best long term operational costs. I drive a 12 year old Lexus ES 300 because it's reliable. If you offer me a choice between something that's slightly cheaper today or has good long term benefits I almost always choose the option which will have the best long term benefits.

I also tend to view the past relatively positively (3.3 past-positive vs. 2.8 past-negative). On the present measurements I score very low on the present-fatalistic measurement (1.2) and moderately on hedonistic (3.7). I'm sure that I could improve my perception of the past to improve my overall outlook – and I know that I can get better at living in the moment (present-hedonistic) – but that's actually one of the points of the book. Once you know what your time perspectives are you can work to gently nudge them into a more healthy perspective. (At least more happy, see my book review of Stumbling on Happiness.)

For me the book provides a context for evaluating my own perspectives and for understanding why other folks might not understand the inherent benefits of solutions that have long term rewards. If you're interested in psychology, how you think or how to relate to others I think you'll like The Time Paradox – The New Psychology of Time That Will Change Your Life.

Categories: Book Review, Professional | 1 Comment

Categories

Links

Archives

Other Blogs