The PMF Program should have completed the in-person assessments today. Up to this point, I've been mostly silent on the issue because I didn't want to risk influencing the assessment process. Now that it's finished, I think it's fine for everyone to share their experiences and talk more freely about how they think they did. Include as few or as many details as you are comfortable sharing.
The next update, the one where we find out who is a finalist, will be sometime in January. In the mean time, let's hear from you all.
Showing posts with label in-person assessment. Show all posts
Showing posts with label in-person assessment. Show all posts
Friday, December 16, 2011
Friday, May 20, 2011
Question: Where Should the In-Person Assessments Have Occurred?
This is definitely something exploratory, and it's certainly not intended to be predictive. I am just musing here, partially to determine whether the in-person assessment locations were sound or not.
So the question is this: Based on what we know of the 2011 semifinalist pool, where would the most effective in-person assessment locations have been? Let's broaden that a bit to get at what we really might want to know. Which cities with a Federal presence would have made good cities in which to conduct PMF in-person assessments, based on what we know about where the semifinalists (those invited to the in-person assessments) presumably originated? I hope that's a clear question, but we can break this down into components so the criteria are better defined, and the follow-up questions are enumerated.
One thing you'll notice, of course, is that Washington, DC, is listed here, even though it's not an FEB location. I trust you'll understand why this is the case. Regardless, what we see is that, outside Washington, DC, the top 5 FEB locations are Boston (161), Atlanta (118), New York City (110), Chicago (92), and San Francisco (76). If we were looking for validation of the 2011 in-person assessment location choices, this might suffice. What the top 5 FEB list doesn't really account for, though, is that there are significant numbers in other locations. The trick here would be to determine locations that are central to a region in some way. DC makes sense for most of the East Coast, especially given the ease of transportation between, for instance, Boston and DC. It is entirely fitting, then, to keep DC as an assessment location. Atlanta also makes sense for large portions of the South. The Midwest is well served by Chicago, and the West Coast is well served by San Francisco (although Los Angeles looks to be a good second choice). That just leaves areas like the North Plains and the Southwest less well served. But we can frame a different question that might help here. Let's eliminate all but one NE city (DC), one Southern city (Atlanta), one Midwest city (Chicago) and one Western city (San Francisco), leaving the others on the list to see what we can come up with. That leaves us with the following:
Based on this, I think we can recommend that either a city in Texas or Oklahoma City could serve as the only other location needed. I am choosing OKC because of its fairly central location compared to Denver and Albuquerque. If we do that, then the numbers look like this:
Even so, the payoff for adding OKC is much lower than other locations, and it may not ultimately be worth the effort to add the assessment location.
Next, let's see if we can determine whether there's a distance factor involved here. That is, if we take these five locations, is there an average distance we're looking for that might be ideal? The first table shows us a widely variable average distance between the school and the assessment center.
The largest average is for DC, but this can be partially explained by the large distribution of semifinalists (Boston to a point about halfway between Atlanta and DC) and the inclusion of overseas schools in this list, all of which are in excess of 2000 miles away. There aren't many, but they are enough to affect the result. Filtering those out will give us perhaps something more meaningful.
So there you have it. These are pretty good distances from what I can tell, but I am interested to know what you all think. As one final point of comparison, here are the average distances to the original in-person assessment locations. By omitting OKC, we raise the averages for Chicago, San Francisco, and Atlanta, but DC is unaffected.
Conclusion
So after examining the locations that might have made sense for the in-person assessment, what we found is that the original locations seemed to be about right. We could fragment the assessment centers a bit more by adding one in OKC, but doing much more than that seems to have a lower benefit. What do you all think? Were these distances doable for you? I know many of you would have preferred not to travel as far as you did, but consider the alternatives (such as all in-person assessments being held in DC). What locations do you think should be considered?
So the question is this: Based on what we know of the 2011 semifinalist pool, where would the most effective in-person assessment locations have been? Let's broaden that a bit to get at what we really might want to know. Which cities with a Federal presence would have made good cities in which to conduct PMF in-person assessments, based on what we know about where the semifinalists (those invited to the in-person assessments) presumably originated? I hope that's a clear question, but we can break this down into components so the criteria are better defined, and the follow-up questions are enumerated.
- Federal presence: This can mean one of a few things. First, and fundamentally, is there any Federal agency with an office in the city? For many cities, the answer is a qualified yes. Qualified, because even if a city has some Federal presence, this does not mean that the city is at all suited to hosting in-person assessments, either because it is located too far away from the bulk of semifinalists, or because it is simply too small to provide convenient transportation options. It turns out there is another way we can measure Federal presence in a city: The Federal Executive Boards (FEB). FEBs form a nationwide network of Federal branches providing communication and collaboration solutions to agencies outside the DC area. Given their wide geographic distribution, it seems clear that FEB cities might serve as a good starting point to analyze future in-person assessments. In the graphics below, I show a summary view of how many semifinalists were listed as closest to each of these cities.
- Semifinalists: I chose semifinalists from this year 1) because there were semifinalists for 2011, and 2) because semifinalists were the ones invited to take in-person assessments. It doesn't make much sense to me to choose nominees or finalists for this particular comparison, although choosing nominees would at least provide some information for future planning. What we know about semifinalists is the school they listed in their materials, and not much more. This is a limitation of the data set, of course, but it's all we have to work with. What we have to assume from it is that the schools in question were correctly identified for purposes of geolocation; that every semifinalists were correctly listed with their schools; and that the locations of the schools reflect the geographic origins of the semifinalists. That's a tall order, but again, what choice do we have? Some of these schools conduct extensive online programs that mean students could be widely dispersed beyond the brick-and-mortar campus. What we have, then, is close enough approximation of the truth for this analysis.
One thing you'll notice, of course, is that Washington, DC, is listed here, even though it's not an FEB location. I trust you'll understand why this is the case. Regardless, what we see is that, outside Washington, DC, the top 5 FEB locations are Boston (161), Atlanta (118), New York City (110), Chicago (92), and San Francisco (76). If we were looking for validation of the 2011 in-person assessment location choices, this might suffice. What the top 5 FEB list doesn't really account for, though, is that there are significant numbers in other locations. The trick here would be to determine locations that are central to a region in some way. DC makes sense for most of the East Coast, especially given the ease of transportation between, for instance, Boston and DC. It is entirely fitting, then, to keep DC as an assessment location. Atlanta also makes sense for large portions of the South. The Midwest is well served by Chicago, and the West Coast is well served by San Francisco (although Los Angeles looks to be a good second choice). That just leaves areas like the North Plains and the Southwest less well served. But we can frame a different question that might help here. Let's eliminate all but one NE city (DC), one Southern city (Atlanta), one Midwest city (Chicago) and one Western city (San Francisco), leaving the others on the list to see what we can come up with. That leaves us with the following:
Based on this, I think we can recommend that either a city in Texas or Oklahoma City could serve as the only other location needed. I am choosing OKC because of its fairly central location compared to Denver and Albuquerque. If we do that, then the numbers look like this:
Even so, the payoff for adding OKC is much lower than other locations, and it may not ultimately be worth the effort to add the assessment location.
Next, let's see if we can determine whether there's a distance factor involved here. That is, if we take these five locations, is there an average distance we're looking for that might be ideal? The first table shows us a widely variable average distance between the school and the assessment center.
The largest average is for DC, but this can be partially explained by the large distribution of semifinalists (Boston to a point about halfway between Atlanta and DC) and the inclusion of overseas schools in this list, all of which are in excess of 2000 miles away. There aren't many, but they are enough to affect the result. Filtering those out will give us perhaps something more meaningful.
So there you have it. These are pretty good distances from what I can tell, but I am interested to know what you all think. As one final point of comparison, here are the average distances to the original in-person assessment locations. By omitting OKC, we raise the averages for Chicago, San Francisco, and Atlanta, but DC is unaffected.
Conclusion
So after examining the locations that might have made sense for the in-person assessment, what we found is that the original locations seemed to be about right. We could fragment the assessment centers a bit more by adding one in OKC, but doing much more than that seems to have a lower benefit. What do you all think? Were these distances doable for you? I know many of you would have preferred not to travel as far as you did, but consider the alternatives (such as all in-person assessments being held in DC). What locations do you think should be considered?
Monday, February 28, 2011
PMF 2011 In-Person Assessment: Debrief
[Also posted here]
The final make-up assessments were (presumably) completed on Friday, February 25, 2011.
In the interest of me not having to publish all the queued comments, I am opening a new thread where you can discuss your experience with the in person assessment this year. What did you think? How do you think you did? How do you think your fellow semifinalists did? What percentage of no-shows did you see?
We should be able to discuss the details of the assessment in much more depth now that the assessments are all complete. I am interested in what you thought of the questions, the subject matter covered, the assessors, the assessment centers, all of it.
Soon I will follow up with two more posts. One will be an open thread for you to discuss the finalist results, which of course are still forthcoming. The other will be an in-depth examination of the process from the assessor's point of view. Let me know if you want any other topics and I will try to accommodate.
The final make-up assessments were (presumably) completed on Friday, February 25, 2011.
In the interest of me not having to publish all the queued comments, I am opening a new thread where you can discuss your experience with the in person assessment this year. What did you think? How do you think you did? How do you think your fellow semifinalists did? What percentage of no-shows did you see?
We should be able to discuss the details of the assessment in much more depth now that the assessments are all complete. I am interested in what you thought of the questions, the subject matter covered, the assessors, the assessment centers, all of it.
Soon I will follow up with two more posts. One will be an open thread for you to discuss the finalist results, which of course are still forthcoming. The other will be an in-depth examination of the process from the assessor's point of view. Let me know if you want any other topics and I will try to accommodate.
Monday, January 10, 2011
2011 PMF In-Person Assessment: Locations
[Also posted on GovLoop]
In-person assessments for the 2011 PMFs begin on Tuesday, January 18 and run through Friday, February 18 at four locations throughout the U.S. According to the PMF Program Office, all notifications have gone out, so all 1530 of you should be scheduled for an assessment date and location (assuming you went ahead with it). The next step is getting to the assessment location.
The locations are as follows (also available on the PMF Program's Facebook page here):
Washington DC:
View Larger Map
Embassy Suites Downtown
1250 22nd St NW
Washington, DC 20037
Atlanta, GA:
View Larger Map
Atlanta Marriott Suites
35 14th Street NE
Atlanta, GA 30309
Chicago, IL:
View Larger Map
John C. Kluczynski Federal Building
230 South Dearborn Street
Suite 3070
Chicago, IL 60604
San Francisco, CA:
View Larger Map
San Francisco Federal Building
90 Seventh Street
B-150
San Francisco, CA 94103
I can't speak for the other cities, but I can at least tell you that public transportation directions are not available for Washington, DC, directly from Google Maps (they are, but aren't accurate). Use Google Maps to orient yourself and then use wmata.com to get routes and timetables. Any advice for the other cities would be appreciated.
In-person assessments for the 2011 PMFs begin on Tuesday, January 18 and run through Friday, February 18 at four locations throughout the U.S. According to the PMF Program Office, all notifications have gone out, so all 1530 of you should be scheduled for an assessment date and location (assuming you went ahead with it). The next step is getting to the assessment location.
The locations are as follows (also available on the PMF Program's Facebook page here):
Washington DC:
View Larger Map
Embassy Suites Downtown
1250 22nd St NW
Washington, DC 20037
Atlanta, GA:
View Larger Map
Atlanta Marriott Suites
35 14th Street NE
Atlanta, GA 30309
Chicago, IL:
View Larger Map
John C. Kluczynski Federal Building
230 South Dearborn Street
Suite 3070
Chicago, IL 60604
San Francisco, CA:
View Larger Map
San Francisco Federal Building
90 Seventh Street
B-150
San Francisco, CA 94103
I can't speak for the other cities, but I can at least tell you that public transportation directions are not available for Washington, DC, directly from Google Maps (they are, but aren't accurate). Use Google Maps to orient yourself and then use wmata.com to get routes and timetables. Any advice for the other cities would be appreciated.
Wednesday, December 22, 2010
2011 PMF In-Person Assessment: Open Thread
[Also posted here]
Now that the 2011 Presidential Management Fellows semifinalists have been chosen and the PMF Program Office has posted the list, it's time to think about the next hurdle, the in-person assessment. 1530 people (congratulations to you!) will convoke in four cities around the country between mid-January and late-February for a day-long assessment that includes an individual presentation, a group exercise, and a proctored writing exercise.
This post, and the comment thread, can be used to share assessment preparation tips as well as general tips on navigating the cities in question, so hotel recommendations, flight options, eateries, and in-city transit means are up for discussion. If you would kindly indicate at the beginning of any city-specific information comments which city you are talking about, that would be helpful for people scanning through the comments.
A note on assessment tips: Let's do our best to preserve the integrity of the assessment process, at least during the assessment window. If, after the assessment window closes, you wish to share your experience in more detail for future applicants, I heartily invite you to do so. Remember that while this assessment window is open, you are effectively competing with one another; plan your strategy accordingly, especially if people start sharing detailed experiences. This, in fact, is the one major drawback I see to conducting this style of assessment: information provided by early testers may have an impact on the results of later testers. Note that this thread can also serve as a place to discuss the merits and drawbacks of an in-person assessment.
Please share with us your plans for the assessment: time and location you've selected, preparation tips or guides you think might be helpful, and thoughts on the process so far.
Now that the 2011 Presidential Management Fellows semifinalists have been chosen and the PMF Program Office has posted the list, it's time to think about the next hurdle, the in-person assessment. 1530 people (congratulations to you!) will convoke in four cities around the country between mid-January and late-February for a day-long assessment that includes an individual presentation, a group exercise, and a proctored writing exercise.
This post, and the comment thread, can be used to share assessment preparation tips as well as general tips on navigating the cities in question, so hotel recommendations, flight options, eateries, and in-city transit means are up for discussion. If you would kindly indicate at the beginning of any city-specific information comments which city you are talking about, that would be helpful for people scanning through the comments.
A note on assessment tips: Let's do our best to preserve the integrity of the assessment process, at least during the assessment window. If, after the assessment window closes, you wish to share your experience in more detail for future applicants, I heartily invite you to do so. Remember that while this assessment window is open, you are effectively competing with one another; plan your strategy accordingly, especially if people start sharing detailed experiences. This, in fact, is the one major drawback I see to conducting this style of assessment: information provided by early testers may have an impact on the results of later testers. Note that this thread can also serve as a place to discuss the merits and drawbacks of an in-person assessment.
Please share with us your plans for the assessment: time and location you've selected, preparation tips or guides you think might be helpful, and thoughts on the process so far.
Friday, December 17, 2010
2011 PMF Semi-Finalists and In-Person Assessment: Open Thread
[Cross-posted to GovLoop here]
No, the semi-finalist list is not out (to my knowledge). Once it does come out, I'm sure most of you will know before I do. (Nor, as an aside, has the PMF site finally cleared what I assume is legal review for publishing). This post can serve as an open thread for anyone who wants to share the good or bad news, once it arrives.
In the mean time, however, we can discuss the in-person assessment a bit. A commenter in a previous thread pointed me to a document proposed to (but not, apparently, published by) the Chief Human Capital Officers (CHCO) Council regarding the administration of the in-person assessment (warning: PDF). Based on other information I have seen, the document appears to cover the breakdown of the in-person assessments, anticipating the number of semi-finalists that are expected at each of the assessment centers. Of particular note is the anticipated number of semi-finalists: 1500. Of the over 9100 applicants and 7400 ultimately nominated, the PMF Program Office intends to invite around 1500 semi-finalists to the in-person assessments. For those interested in graphing this, that's: 82% of applicants become nominees; and 20% of nominees (or 16% of applicants) will be chosen as semi-finalists. Further, the PMF Program Office indicates that about 750-800 semi-finalists will advance to finalist status (as a percentage of applicants, that's 8.2-8.7%; as a percentage of nominees, that's 10-10.7%; and as a percentage of semi-finalists, that's 50-53%). Historically, agencies have been able to absorb maybe half of the finalists, depending on demand. It's a useful approximation here, but given the state of the economy and the political climate toward federal employees right now, it's anyone's guess as to how this will play out in 2011. If we take the historical rate, then, an applicant has about a 1-in-25 shot at landing a position; a nominee improves to 1-in-20; and a semi-finalist improves to 1-in-4. Put another way, the program selects only about 4-5% of its applicants, which is pretty competitive (but you knew this, right?)
Now back to the in-person assessment. The locations (which were apparently polled preferentially) are:
(According to the document) The assessment is a one-day, day-long assessment before a three member panel (comprised of an OPM official, an official from another agency, and a current or former PMF), and it will require semi-finalists to prepare and deliver an individual presentation, participate in a group exercise, and deliver a proctored writing assessment. The assessments will be spread out over approximately three weeks, so you will have a time window in which to schedule the assessment.
What do you think of this format? The assessment locations? When/where would you plan (or are you planning) to take the in-person assessment? And finally, is this process something in which you would expect to participate next year (or thereafter), should you become a finalist and find an appointment?
No, the semi-finalist list is not out (to my knowledge). Once it does come out, I'm sure most of you will know before I do. (Nor, as an aside, has the PMF site finally cleared what I assume is legal review for publishing). This post can serve as an open thread for anyone who wants to share the good or bad news, once it arrives.
In the mean time, however, we can discuss the in-person assessment a bit. A commenter in a previous thread pointed me to a document proposed to (but not, apparently, published by) the Chief Human Capital Officers (CHCO) Council regarding the administration of the in-person assessment (warning: PDF). Based on other information I have seen, the document appears to cover the breakdown of the in-person assessments, anticipating the number of semi-finalists that are expected at each of the assessment centers. Of particular note is the anticipated number of semi-finalists: 1500. Of the over 9100 applicants and 7400 ultimately nominated, the PMF Program Office intends to invite around 1500 semi-finalists to the in-person assessments. For those interested in graphing this, that's: 82% of applicants become nominees; and 20% of nominees (or 16% of applicants) will be chosen as semi-finalists. Further, the PMF Program Office indicates that about 750-800 semi-finalists will advance to finalist status (as a percentage of applicants, that's 8.2-8.7%; as a percentage of nominees, that's 10-10.7%; and as a percentage of semi-finalists, that's 50-53%). Historically, agencies have been able to absorb maybe half of the finalists, depending on demand. It's a useful approximation here, but given the state of the economy and the political climate toward federal employees right now, it's anyone's guess as to how this will play out in 2011. If we take the historical rate, then, an applicant has about a 1-in-25 shot at landing a position; a nominee improves to 1-in-20; and a semi-finalist improves to 1-in-4. Put another way, the program selects only about 4-5% of its applicants, which is pretty competitive (but you knew this, right?)
Now back to the in-person assessment. The locations (which were apparently polled preferentially) are:
- Washington, DC
- Chicago, IL
- San Francisco, CA
- Atlanta, GA
(According to the document) The assessment is a one-day, day-long assessment before a three member panel (comprised of an OPM official, an official from another agency, and a current or former PMF), and it will require semi-finalists to prepare and deliver an individual presentation, participate in a group exercise, and deliver a proctored writing assessment. The assessments will be spread out over approximately three weeks, so you will have a time window in which to schedule the assessment.
What do you think of this format? The assessment locations? When/where would you plan (or are you planning) to take the in-person assessment? And finally, is this process something in which you would expect to participate next year (or thereafter), should you become a finalist and find an appointment?
Subscribe to:
Posts (Atom)