I hope this is held again in 2012 and that Oracle sends more developers to the conference. I had useful conversations with the four developers and development managers who attended from Oracle. I also had great discussions with Percona, Monty Program, Continuent and many others. At home I sit and type, at the conference I stand and talk. It is a welcome change.The MySQL team has been very productive recently with many bug fixes, many features implemented, a great 5.5 release and what appears to be an even better 5.6 release. The changes in 5.6 are a really big deal. I can't wait to stop porting rpl_transaction_enabled to get crash-proof slave state.The value-added community continues to push the state of the art for those who can't wait for the great new features to be GA or for those with special problems.  I am interested in trying parallel replication apply from Tungsten, working with Monty Program to improve monitoring in MariaDB and working with Percona to improve InnoDB quality-of-service for high-throughput OLTP.I have begun to catch up on my reading to figure out what has changed for 5.6. I probably need another week to finish reading the many useful blogs and presentations. DimitriK published a 5-part performance report that I have yet to start (1, 2, 3, 4, 5). I am happy to find that few more MySQL developers at Oracle have begun blogging (Oystein, Didrik, Luis, Olav). How do I type the accented "O" in Oystein?First the InnoDB changes:Information schema system tables - InnoDB has moved a lot of data to IS tables. The ones listed here are not the most interesting to me but the migration in general makes life easier for me. I don't know if these are new in 5.6, but they are more interesting to me than the ones described in the InnoDB blog: INNODB_BUFFER_PAGE, INNODB_BUFFER_PAGE_LRU, and INNODB_BUFFER_POOL_STATS.No more kernel_mutex - Wow! This is a huge deal for multi-core performance and will enable even more improvements in future releases. As part of the admission control feature we have been trying to reduce kernel_mutex contention and noticed that 5.5 already was much better for that.Persistent index cardinality statistics - when a server is restarted all index cardinality stats are computed in MySQL 5.1 and without the Facebook patch, this is serialized because of LOCK_open. That can create too many stalls on restart. I assume this allows a DBA to populate the stats table manually which can be very useful for a deployment with many scale-out slaves to prevent query plans from changing between slaves. Other details are at Oystein's blog. Multi-threaded purge - I want this right now. This and parallel replication apply make it possible to deploy large databases on slaves using disk setups that match what is available on a master. Otherwise you must read Yoshinori's slides to find the workaround.Data dictionary LRU helps if you have a lot (or too many) tables.Page cleaner thread - This is a good step but I wonder if it is enough and need to read code or wait for a Percona performance report. Has anything been done to prevent stalls when the async flush tries to flush too many pages in one call? The problem is that all dirty pages from an extent are flushed when at least one page with a too-old LSN is in the extent. Flushing neighbor pages can increase the number of pages flushed by up to 64X. I have seen benchmark servers stall for 60+ seconds while 200,000+ pages were flushed when the async limit was reached. This can be very painful in servers that are able to cache the database and fill with many dirty pages.memcached API for InnoDB - this should enable HandlerSocket like performance while supporting an API that is already supported by most clients (PHP, Java, Python, ...). I think that non-SQL interfaces to InnoDB will be a big deal.metrics table for InnoDB counters - I spent too much time adding counters to SHOW STATUS for InnoDB. Now you don't have to.The InnoDB team has done a great job of making it easy to understand the changes. Getting docs for the replication changes required a bit more searching as they were not referenced from the main Oracle announcement. I added links to worklog entries and blogs by Mats and Luis. I am very interested in many of the replication changes that match and exceed what was in the Google patch.parallel replication apply - I am confused. The labs page and worklog state that this is for RBR. I want parallel apply for SBR to overcome slave replication lag from IO bound slaves. Parallel apply provides parallel IO requests. Per the blog by Luis, I think this supports SBR.system tables for slave state - I don't know if WL2775 describes what was implemented. Feedback from gmaxia makes me wish for more docs. Hopefully I can stop porting rpl_transaction_enabled and begin to use this instead. I need to read the blog by Mats.replication checksums - I want binlog event checksums. I have not needed them for a long time since a certain bug was fixed but I would rather not worry about that problem again. Mats also wrote about this.informational log events - I need the original SQL including its query comment in the binlog if I am to use RBR. With this feature I have one less excuse for not trying RBR.remote binlog backup - We already have this in the 5.1 Facebook patch thanks to a backport by Harrison. It lets you archive the binlog almost as soon as it is written. universal group identifiers - This might be the equivalent of global transaction IDs from the Google patch. optimized RBR logging - This will be a big deal for tables with BLOB columns that get frequent updates to non-BLOB columns. I have a few of those.Time delayed replication - I don't need this but many others will.There also appear to be many useful changes in the Performance Schema. Alas, it isn't easy to figure out what has changed from reading the big announcement. I haven't been a fan of the P_S given my primary need is for aggregated workload stats (per-table, per-index, per-user) and that the user_stats patch from Facebook, Percona, MariaDB and Google has had that for many years in a form that is trivial to use (no setup) with very low overhead (it is always enabled in my benchmarks) and will always be enabled in production. It also is very easy to use -- select * from easy_to_remember_table_name. I hope the P_S can also provide that. Mark and Marc have begun to write more about this. I hope that continues.

Why do B-trees need “Tricks” to work? Marko Mäkelä recently posted a couple of “tips and tricks” you can use to improve InnoDB performance. Tips and tricks. A general purpose relational database like MySQL shouldn’t need “tips and tricks” to perform well, and I lay the blame on design choices that were made in the early ’70s: the B-tree data structure underlying all OldSQL databases. B-trees were designed for machines that had very different performance characteristics than the machines of today. Hardware has changed, but B-trees are the same. Tips and Tricks are an attempt to make up the difference. So B-tree implementers — InnoDB, Oracle, MS SQL Server — are fighting an uphill battle; they’re fighting the future. B-trees just aren’t meant to cope with high-bandwidth, slow-seek-time storage systems, because they perform unnecessary disk seeks. In fact, they aren’t meant to cope with lower-bandwidth, fast-seek-time storage systems (like SSDs), because they waste bandwidth. Hardware trends move computers further and further from B-trees’ sweet spot. In 1972, they were a great idea, but a lot has changed. Don’t get me wrong. InnoDB is a great piece of software. In my own testing, its B-tree implementation outperforms the big enterprise implementations, though it lacks some features like hot indexing and hot column addition and deletion. So the problem is OldSQL B-trees, which have two big shortcomings. These are both illustrated very well in the tips and tricks posting. They are slow to index, and they fragment. Let’s look at Marko’s points one by one: Marko’s first point is that “Data Dictionary Language (DDL) operations have traditionally been slow in MySQL”. The slow part of DDL is mostly ALTER TABLE commands. Hot Column Addition and Deletion and Hot Indexing, recently released in TokuDB 5.0, brings the downtime of these schema changes from hours to seconds. Marko’s second point is that insertions, deletions and updates “can be slow [with InnoDB] even despite the change buffering in MySQL 5.5.” Fast insertions, deletions and updates have been a hallmark of TokuDB’s Fractal Tree Indexes since they were introduced. Marko’s next point is that if you populate an InnoDB index out of order, it will fragment. His proposed solution involves dropping and adding indexes, which means table downtime. The best way to deal with fragmentation is to not fragment your tables in the first place. So what’s the solution? You guessed it: TokuDB tables don’t fragment by design, so this particular chunk of maintenance is not needed. Finally, Marko discusses fill factors in B-trees, and the impact of packed versus unpacked nodes on future modifications. TokuDB is based on cache-oblivious analysis, which means that you don’t need to worry about fill factors, and it means that there are very few parameters to set. In fact, we recommend that you leave the parameters set to the defaults. Treats, not tricks In short, TokuDB’s Fractal Tree indexes are fast to update (20x-80x faster than InnoDB) and never fragment. Hot schema changes, fast insertions, no fragmentation, and no tuning. That’s what happens when your solution matches your hardware. We have sessions coming up at both O’Reilly and Collaborate 11 conferences next week, as well as a booth at O’Reilly. Please come by and tell us your “tips and tricks” (and band-aids and duct tape) for working around old database structures, and we’ll work to get you on a better path with Tokutek’s NewSQL alternative.

People's Credit Company, an independently owned finance company located in beautiful NE Portland, Oregon, is recruiting for a full time Web Developer to join our team.

We understand that the contributions of each & every employee is key to our success. We don’t limit any position solely to its assigned responsibilities. This job description is merely an outline of primary duties, qualifications and job scope. We want employees who are comfortable offering their services & ideas whenever necessary to ensure the success of all our endeavors.

The Job:
The Senior Programmers position is responsible for design, development, testing, deployment and maintenance of internet and CRM ColdFusion applications by performing specific responsibilities:
• Develop code, test/debug code, and provide full development life-cycle support
• Drive designs for database schema and tables for SQL 2000
• Document functional specifications, design, procedures, unit test, integrated system test and end user acceptance test plans and application support manuals
• Build, code, and test complex application features and technical designs

The Scope:
Leverage your experience, foresight, and analytical skills to find solutions and complete tasks. Duties involve a fairly high degree of complexity due to the nature of our mature CRM/Website product. Work is performed independently according to established guidelines; however a certain degree of creative latitude and discretion are important. While this position reports directly to the Chief Technology Officer, there is interaction both within and outside the company. We want you to want to work effectively with all business units to meet deadlines and goals.

The Essentials:
• ColdFusion 7 or better
• Solid understanding of HTML, CSS, JavaScript, and AJAX
• Strong documentation and debugging/QA skills
• Excellent communication skills and effective teamwork
• Proven experience in producing accurate and high quality work
• Ability to independently manage workload and be a self-starter
• High professional ethical standards

Bonus Qualifications:
• Experience with ColdFusion 8, 9 or OpenBD
• Experience with SQL Server 2000, 2005, and/or 2008
• Bachelors Degree in Computer Science or related field from an accredited University OR equivalent combination of education and/or experience
• 4+ years experience developing large-scale ColdFusion applications
• Microsoft SQL Server DBA experience
• Experience with payment processor integration preferred (both real-time & batch processing)

The Rest of the Story:
We are a finance company that facilitates it's business via the web. Our web development team is a small group of relaxed professionals who take the job seriously and also like life outside the office. We offer competitive compensation and an excellent benefits package including medical, dental, vacation pay and 401k. This is a full-time job located in NE Portland, Oregon. This is NOT a telecommute position.

To apply:
We are an equal opportunity employer building a diverse workplace. If you are seeking a small employee friendly corporation, please submit your application and resume on line at http://www.ocmusa.com/jobs/ Click “Click Here For Job Applications”, scroll to “People's Credit” Click “Apply (Login)” for “Programmer - Portland Oregon”.”Create a profile” Fill out application and submit.

People's Credit Company, an independently owned finance company located in beautiful NE Portland, Oregon, is recruiting for a full time Web Developer to join our team.

We understand that the contributions of each & every employee is key to our success. We don’t limit any position solely to its assigned responsibilities. This job description is merely an outline of primary duties, qualifications and job scope. We want employees who are comfortable offering their services & ideas whenever necessary to ensure the success of all our endeavors.

The Job:
The Senior Programmers position is responsible for design, development, testing, deployment and maintenance of internet and CRM ColdFusion applications by performing specific responsibilities:
• Develop code, test/debug code, and provide full development life-cycle support
• Drive designs for database schema and tables for SQL 2000
• Document functional specifications, design, procedures, unit test, integrated system test and end user acceptance test plans and application support manuals
• Build, code, and test complex application features and technical designs

The Scope:
Leverage your experience, foresight, and analytical skills to find solutions and complete tasks. Duties involve a fairly high degree of complexity due to the nature of our mature CRM/Website product. Work is performed independently according to established guidelines; however a certain degree of creative latitude and discretion are important. While this position reports directly to the Chief Technology Officer, there is interaction both within and outside the company. We want you to want to work effectively with all business units to meet deadlines and goals.

The Essentials:
• ColdFusion 7 or better
• Solid understanding of HTML, CSS, JavaScript, and AJAX
• Strong documentation and debugging/QA skills
• Excellent communication skills and effective teamwork
• Proven experience in producing accurate and high quality work
• Ability to independently manage workload and be a self-starter
• High professional ethical standards

Bonus Qualifications:
• Experience with ColdFusion 8, 9 or OpenBD
• Experience with SQL Server 2000, 2005, and/or 2008
• Bachelors Degree in Computer Science or related field from an accredited University OR equivalent combination of education and/or experience
• 4+ years experience developing large-scale ColdFusion applications
• Microsoft SQL Server DBA experience
• Experience with payment processor integration preferred (both real-time & batch processing)

The Rest of the Story:
We are a finance company that facilitates it's business via the web. Our web development team is a small group of relaxed professionals who take the job seriously and also like life outside the office. We offer competitive compensation and an excellent benefits package including medical, dental, vacation pay and 401k. This is a full-time job located in NE Portland, Oregon. This is NOT a telecommute position.

To apply:
We are an equal opportunity employer building a diverse workplace. If you are seeking a small employee friendly corporation, please submit your application and resume on line at http://www.ocmusa.com/jobs/ Click “Click Here For Job Applications”, scroll to “People's Credit” Click “Apply (Login)” for “Programmer - Portland Oregon”.”Create a profile” Fill out application and submit.

Understanding how well your tables and indexes fit to buffer pool are often very helpful to understand why some queries are IO bound and others not - it may be because the tables and indexes they are accessing are not in cache, for example being washed away by other queries. MySQL Server does not provide any information of this type, Percona Server however adds number of tables to Information Schema which makes this information possible. It is just few queries away: PLAIN TEXT SQL: mysql> SELECT `schema` AS table_schema,innodb_sys_tables.name AS table_name,innodb_sys_indexes.name AS index_name,cnt,dirty,hashed,round(cnt*100/index_size,2) fit_pct   FROM (SELECT index_id,count(*) cnt,sum(dirty=1) dirty ,sum(hashed=1) hashed FROM innodb_buffer_pool_pages_index GROUP BY index_id) bp JOIN innodb_sys_indexes ON id=index_id JOIN innodb_sys_tables ON table_id=innodb_sys_tables.id JOIN innodb_index_stats ON innodb_index_stats.table_name=innodb_sys_tables.name AND innodb_sys_indexes.name=innodb_index_stats.index_name AND innodb_index_stats.table_schema=innodb_sys_tables.schema  ORDER BY cnt DESC LIMIT 20; +--------------+--------------+--------------+------+-------+--------+---------+ | table_schema | table_name   | index_name   | cnt  | dirty | hashed | fit_pct | +--------------+--------------+--------------+------+-------+--------+---------+ | test         | a            | c            | 7976 |     0 |      0 |   13.73 | | test         | a            | PRIMARY      |   59 |     0 |      0 |    0.08 | | sbtest       | sbtest#P#p1  | PRIMARY      |   22 |     0 |      0 |   22.68 | | sbtest       | sbtest#P#p0  | PRIMARY      |   22 |     0 |      0 |   22.68 | | sbtest       | sbtest#P#p2  | PRIMARY      |   21 |     0 |      0 |   21.65 | | sbtest       | sbtest#P#p3  | PRIMARY      |   18 |     0 |      0 |   18.56 | | sbtest       | sbtest#P#p3  | k            |    4 |     0 |      0 |  100.00 | | sbtest       | sbtest#P#p2  | k            |    4 |     0 |      0 |  100.00 | | sbtest       | sbtest#P#p1  | k            |    4 |     0 |      0 |  100.00 | | sbtest       | sbtest#P#p0  | k            |    4 |     0 |      0 |  100.00 | | stats        | TABLES       | PRIMARY      |    2 |     0 |      0 |   66.67 | | stats        | TABLES       | TABLE_SCHEMA |    1 |     0 |      0 |  100.00 | | percona      | transactions | PRIMARY      |    1 |     0 |      0 |  100.00 | +--------------+--------------+--------------+------+-------+--------+---------+ 13 rows IN SET (0.04 sec) This query shows information about how many pages are in buffer pool for given table (cnt), how many of them are dirty (dirty), and what is the percentage of index fits in memory (fit_pct) For illustration purposes I've created one table with partitions to show you will have the real "physical" table name which identifies table down to partition, which is very helpful for analyzes of your access to partitions - you can actually check if your "hot" partitions really end up in the cache and "cold" are out of the cache, or is something happening which pushes them away from the cache. You can use this feature to tune buffer pool invalidation strategy, for example play with innodb_old_blocks_pct and innodb_old_blocks_time actually observing data stored in buffer pool rather than using some form of temporary measures. I often check these stats during warmup to see what is really getting warmed up first as well as how buffer pool is affected by batch jobs, alter tables, optimize table etc - the lasting impact these may have on system performance is often caused by impact they have on buffer pool which may take hours to recover. This tool can be also helpful for capacity planning/performance management. In many cases you would learn you need a certain fit to buffer pool for tables/indexes for reasonable performance, you may try to count it too but it may be pretty hard as there are a lot of variables, including page fill factors etc. Entry posted by Peter Zaitsev | No comment Add to: | | | |

A while ago I posted an article on Modifying Microsoft Dynamics GP 2010 Word Templates which demonstrated the basic methods for customizing word templates for Microsoft Dynamics GP 2010.

In that article, I mentioned that if you were using Word 2010 to edit the templates, you would have an option to Maintain compatibility with previous versions of Word on the Save As dialog. See screenshot below:

I had recommended leaving this option checked, but indicated that it did not make much difference.... Well, I was wrong.  The Dynamics GP support and development teams have seen issues with the Template Processing Engine (TPE) when this option is not selected.  The original post has now been updated to clarify that the option must be left selected.

So, what does the option really do?

It turns out that not all .docx files are the same.  There are some differences between Word 2007 .docx format and Word 2010 .docx format.  When this option is selected, the Word 2007 .docx format is maintained.  If you uncheck the option, the document will be saved in Word 2010 .docx format and the Maintain Compatibility option will not be available again for that document.  Once a document has been saved in Word 2010 .docx format, there is no way to save it again in Word 2007 .docx format.

I have a very simple example that can demonstrate this issue.  I added the template POP Purchase Order Blank Form Template based on the POP Purchase Order Blank Form Template* default template.  I then modified it in Word 2010 to add a box (with no fill) around the Creditor/Vendor address. See screenshot below:

I then used Save As to save the template to my Desktop. First saving as POP Purchase Order Blank Form Template with Maintain Compatibility turned on (ie. Word 2007 .docx format), and then as POP Purchase Order Blank Form Template 2010 with Maintain Compatibility turned off (ie. Word 2010 .docx format).  I then imported both templates back into Microsoft Dynamics GP.

When I print using the POP Purchase Order Blank Form Template file everything works as desired:

 However, when I print using the POP Purchase Order Blank Form Template 2010 file, I get the error "The XML data is invalid according to the schema.".

 When I click OK I get a second error "Word found unreadable content ...."

After clicking Yes, the document finally opens.

 

---

 

So, imagine that I am in following situation.  I am using Word 2010 and  I have a template that does not offer the Maintain Compatibility option when I use Save As.  This means that the document has already been saved in Word 2010 .docx format.  I have spent a large number of hours working on this design and don't want to lose my work.  How can I get the template document back to Word 2007 .docx format?

The quick answer is.... start again from the default template and make sure you save it correctly this time. You can imagine that most consultants will not like that answer.

The longer answer is to add a new template based on the default template. Then modify both the newly added template and the template to be "fixed".  Then cut and paste the custom Header, Body and Footer sections into the new template.

Note: When pasting into Word, press delete to remove the additional paragraph mark. Also, remember to change the paper size and margins if necessary.

Once you have the template copied across, use Save As and make sure Maintain Compatibility is selected. Import your "fixed" template back into Microsoft Dynamics GP and the problem is solved.

 

For more information, check out these related posts:

• This Blog: Modifying Microsoft Dynamics GP 2010 Word Templates
   
• Mariano Gomez: Microsoft Dynamics GP 2010 Word Templates summary

Hope you find this fix useful. 

David

One of the interesting facilities in the framework is Configurable Object functionality (it is also known as Task Optimization and also known as Cool Tools). The idea is that any implementation can create their own views of the base product objects and services and implement functionality against those new views. For example, in Oracle Utilities Customer Care and Billing, there is a Person object. That object is used to store and manage information about individuals as well as companies. In the base product you would use the Person Maintenance screen and fill in some of the screen when you wanted to register or maintain and individual as well and fill out other parts of the screen when you wanted to register or maintain a company. This can be somewhat confusing to some customers.

Using Configurable Objects this can be simplified. A business object can be created that is a view of the any object. For example, you could create a Human business object which would cover the aspects of the Person object pertaining to an individual and a Company business object to cover the aspects unique to a company. Even the tag names (i.e. Field Names) in the object can be changed to be more what the implementation is familiar with. The object can also restructure the object. For example, a common identifier for an individual in the USA is the Social Security number, this value is a Person Identifier (as this varies in each country). In the new Human object you can remap the Person Identifier as a Social Security number.

To define a Business Object you use a schema editor built into the browser user interface and use a mapping language to setup the business objects. An example of the language is shown below in an extract of the schema for the Human business object.

As you can see there are mapping as well as formatting and other tags. This information can be built manually or using a wizard which generates the base structure for you to alter. This is all stored as meta data when saved.
Once a Business object is built it can be used as basis for code, other business objects (we support inheritance), called by a screen (called a UI Map) or even as a Web Service.

This is just a start with Configurable Objects as you can also create views of base services called Business Services, Service Scripts used for non-object or complex object processing (as well as other things), UI Maps used for screens and Data Areas to reuse definitions across multiple objects.

Configurable Objects are powerful and I only really touched on them here. Over the next few months I hope to add lots more entries about them.

In this blog post I am going to review Train Signal’s “Exchange Server 2010 Training” DVDs, but before I do I would like to make a disclaimer:

I have sectioned this post off into five main areas. You can jump to them here. There’s a TL;DR summary for those of you who have a server migration to get to and just want the Cliff notes.

• Disclaimer
• My Expectations. Here I explain what my preconceived notions are about the training and what I hope from it. Most of their thoughts were gathered before I experienced the training.
• My Judging Categories. Based on the expectations I have, these are the categories that I will be judging by.
• The Review. Time to put the shoulder-length rubber gloves on because this could to get messy.
• TL;DR. Synopsis.
• The Judgment. I judge the training on a scale of five based on several categories that I made for video training.

Introduction

With SharePoint 2010, you now have the ability to synchronize user profiles between the SharePoint profile database and user repositories such as Active Directory, SUN One LDAP, IBM Tivoli, and Novell eDirectory. If you have a custom user repository such as a database, or any other source, you can still synchronize profiles using Business Data Connectivity.

Figure 1: Types of synchronization connections available in SharePoint 210

 

In this post, I will explain how you can set up profile synchronization between SharePoint 2010 and Sun LDAP Server.

User Profile Synchronization in SharePoint 2010 - Background

User profile synchronization in SharePoint 2010 is what used to be profile import in MOSS 2007 (configured from SSP administration page). The reason why the terminology has been changed is that in MOSS 2007, the user properties of user profiles used to flow one way, from Active Directory (or other LDAP system) to the SharePoint profile database. For example, if a user’s telephone number was changed in Active Directory, the information used to flow into the SharePoint profile database after an import. However, if a user changed their telephone number from their MySite (which shows information from the profile database), the updated telephone number could not flow back to the user’s Active Directory account. In SharePoint 2010, with every profile property, you can configure a synchronization direction (either import or export). If you choose the direction to be “import” the source of the property’s value will be the external system (Active Directory, SUN LDAP etc.), and the property’s value will be copied from the external system to the SharePoint profile database. If you choose the direction to be “export”, the source of the profile property will be the SharePoint profile database. So if a user updates their profile property (such as their picture), that infromation would flow back to the external system after an incremental synchronization.

Figure 2: Configuring synchronization flow in SharePoint 2010

Hope this explains why profile import is now called profile synchronization, now let’s get started with creating a synchronization connection between SharePoint 2010 and SUN LDAP.

Step 1: Collect Information about the SUN LDAP Server

First things first. Before you begin, you will need the following information about your SUN LDAP Server:

• The server name (name of the computer where SUN LDAP is running)
• The port that the LDAP is listening on (By default, LDAP traffic flows over port 389)
• The account name that has the appropriate permissions on the LDAP directory. You will also need the account’s password. Note that you will need the account name in the distinguished name format (e.g. “CN=Directory Manager”)
• The LDAP path of the container that contains the user profiles in the SUN LDAP Server (e.g. OU=People,DC=example,DC=com )
• The exact version of the SUN LDAP Server (e.g. Sun-Directory-Server/7.0 B2009.1104.2330)

Step 2: Register a Trusted Authentication Provider

If not already done, you will need to register a trusted authentication provider. The fact that you are synchronization profiles with SUN One also means that users will probably log into SharePoint using their SUN One accounts and Forms Based Authentication. To allow users to log into SharePoint using their SUN one accounts over FBA, you will need to register an authentication provider, and you will then need to use this authentication provider when creating a synchronization connection. If you are using a trusted claims prvider, you can use the clams provider when creating the synchronization connection.

It is important to understand that profile synchronization does not use the authentication provider at all. Profile synchronization is just interested in knowing the authentication provider that you are using to allow users to log into SharePoint, The reason for this is that when a user logs into a SharePoint site, a profile for the user is created in the SharePoint site collection database.  On the other hand, the profiles that you import from Acitve Directory or SUN One LDAP are stored in the SharePoint profile database. To enable MySites functionality and updated user information across all site collections, the user profiles in the site collection database need to be “linked” to a profile in the profile database. This is the reason why we need to tell profile synchronization about the authentication provider that the profiles should be linked to.

If you have already registered your claims or trusted authentication provider, you can skip the rest of this section and move to the next step. Otherwise, complete the following steps to create a new web application and a trusted authentication provider for the web application:

Registering a trusted authentication provider

1. We will be modifying the web.config of the central administration site so please create a backup copy of the file before modifying so that we can revert back to it in case something goes wrong.
2. Open the web.config file of the central administration web site.
3. Search for the “<membership>”  tag.
4. Between the “<providers>”  and “</providers>” element , paste in the following string:
   <add name="LDAPmembership" type="Microsoft.Office.Server.Security.LdapMembershipProvider, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" server="SunLDAPServerName" port="389" useSSL="false" userDNAttribute="entryDN" userNameAttribute="uid" userContainer="OU=People,DC=example,DC=com" userObjectClass="InetOrgPerson" userFilter="(&amp;(ObjectClass=InetOrgPerson))" scope="Subtree" otherRequiredUserAttributes="sn,givenName,cn" connectionUsername="cn=Directory Manager" connectionPassword="Password1" />
   
   Note that you will need to modify the above string to match your environment. In most cases, you will need to modify these attributes: server, userContainer, connectionUsername and connectionPassword. Use the information collected in step 1.
5. Save and close web.config file
6. Open the central administration web site
7. From the left navigation bar, click on “Application Management”
8. Under “Web Applications”, click on “Manage Web Applications”
9. From the Ribbon menu, click on “New”
10. Choose “Claims Authentication” as the authentication type
11. Select “Enable Forms Based Authentication” and type in the name of the membership provider you added to web.config in step 4, as shown in the following screenshot:
    
12. Fill in all other required values and click “OK”. After the web application gets created, your trusted authentication provider will also get registered.

Step 3: Add the registry entry to the server running the profile synchronization service

In step 1, I had mentioned that you will need to collect the exact build version of the SUN LDAP Server. We will be using that information in this step. If the version of SUN LDAP server is 5.x or lower, you can skip this step and move on to the next step.

The version of SUN LDAP server that I am using is 7.0 (exact build information obtained in step 1: “Sun-Directory-Server/7.0 B2009.1104.2330”). Please log on to the server in your SharePoint farm where the “Forefront Identity Manager Synchronization Service” is running (which is basically FIM), as shown in the following screenshot:

 

On this server, you will need to edit the registry and add the exact build information of the SUN LDAP Server. Details on how.where the registry entry needs to be made can be found at http://support.microsoft.com/kb/2290443 . Excerpt from the article is provided below:

To resolve this problem, create the following registry key and specify the version of the Sun Java System Directory Server you are attempting to connect to.
Click Start, click Run, type Regedt32, and then click OK.
Locate and then right-click the following registry key:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\FIMSynchronizationService\Parameters
Click New, and then click Multi-string Value.
Set the following parameter values:
Value name: iPlanetMASupportedServers
Value type: (REG_MULTI_SZ)
Set the value of the registry key to the following value:
Sun-Java(tm)-System-Directory/6.3 B2008.0311.0946
Note The precise format of the value will vary in different versions of Sun ONE. Be sure to use the correct version number for your environment.

Step 4: Create the synchronization connection

In this step, we will create the synchronization connection.

1. From the central administration web site, browse to the user profile service application management page:
   
   
2. Click on “Configure Synchronization Connections”
3. Click on “Create New Connection”
4. Type in the name of the connection
5. In the connection type, select “Sun Java System Directory Server”
6. In “Directory Service Server Name”, type in the name of the computer where SUN LDAP Server is running (use the information collected in step 1).
7. This step is a little tricky.In this step, you will need to select the trusted authentication provider that you would like to associate to the user profiles. This is the same authentication provider that you registered in Step 2. In the “Authentication Provider Type” drop down, select “Trusted Claims Provider Authentication”. If you don’t see your authentication provider in the “Authentication Provider Instance” drop down, select “Forms Authentication” from the “Authentication Provider Type” drop down. Now you should see the authentication provider that you registered in step 2:
   
   
   
   HINT: If you don’t see your authentication provider instance in the drop down, toggle the “Authentication Provider Type” to Claims and then back to forms authentication.  This should re-populate the list.
8. In the “Account Name” text box, type in the distinguished name of the account that has the appropriate permissions on the SUN LDAP Directory (this information was collected in step 1. Example: CN=Directory Manager).  
9. Type in the password and the password confirmation for the account
10. Type in the port that the LDAP server is listening on
11. Click on “Populate Containers”. Wait for the containers to be populated and then select the container that contains the user profiles. NOTE: If you get an error when you click on “Populate Containers”, verify that the information you entered in Server Name, Account Name, Password and Port is correct.
12. Click OK

After the synchronization connection gets created, go ahead start a full synchronization and you should see user profiles getting synchronized.

Troubleshooting

1. You get the “Unable to process Create message” error when creating the synchronization connection
   
   This is a known issue that occurs if you are running Sun 6.x or higher and have not added the registry entry correctly as mentioned in step 3 (Add the registry entry to the server running the profile synchronization service). To resolve this issue, please verify that you have the correct build information about the SUN LDAP Server and edit the registry entry to reflect the correct information. For more information about this problem and the solution, please review http://support.microsoft.com/kb/2290443
   
2. You get the “Unable to process Put message” error when creating the synchronization connection
   Please review the application event log of the server where the Forefront Identify Manager Synchronization Service is running. If you see the below error in the event log, you will need to request a configuration change in the SUN LDAP server:
   
   Log Name:      Application
   Source:        FIMSynchronizationService
   Date:          11/21/2010 7:48:34 PM
   Event ID:      6306
   Task Category: Server
   Level:         Error
   Keywords:      Classic
   User:          N/A
   Computer:      tehnoonr-WS08-5.tehnoonr01.lab
   Description:
   The server encountered an unexpected error while performing an operation for the client.
   "ERR: MMS(3564): amparse.cpp(1765): AM: attribute 'givenName' not found in source schema
   BAIL: MMS(3564): amparse.cpp(1765): 0x80230516 (The attribute mapping rules XML defines an invalid/incomplete rule.)
   
   The above error occurs if the attribute “nsslapd-return-exact-case” is set to “off” on the SUN LDAP Server.  In order to successfully create a synchronization connection between SharePoint and SUN LDAP Server, you will need to set this attribute to “on” while you create the synchronization connection. Once the synchronization connection has been created, you can set this attribute back to “off”. Setting this attribute to “off” causes all the attribute names to be returned in lower case, and SharePoint requires the attributes to be returned per the RFC 2256 standard during connection creation. 
   
3. Profile synchronization gets completed but no user profiles get added to the profile database
   This is section covers a rather broad range of issues that you may be running into. If the profile synchronization completed but you don’t see any profiles, you should look for errors in the application event log of the server where the Forefront Identify Manager Synchronization Service is running. You can also bring up the FIM client (miisclient.exe from C:\Program Files\Microsoft Office Servers4.0\Synchronization Service\UIShell) and see if you find any errors in the latest runs (Operations Tab):
   
   
   
   If you don’t find any errors that stand out, there could be two reasons why the profiles are not coming in:
   
   a) The account that you are using to synchronize profiles (one you entered when creating the synchronization connection) does not have the required permissions on the SUN LDAP Server. The account needs to read certain attributes on the LDAP server including nsslapd-sizelimit attribute in cn=config container and lastchangenumber , nsUniqueId and numSubordinates attributes at the root of the container from which user profiles are being synchronized. The account should be able to read all these attributes in the SUN LDAP Server.
   
   b) You have created custom structural object classes in your SUN LDAP Server and associated the custom structural class to the users. In order to add additional attributes to user objects in SUN LDAP, you can create auxiliary or structural object classes. If you create structural object classes and associate them to user objects in SUN LDAP, the default synchronization connection that you created in SharePoint will not be able to synchronize those users. Additional configuration is required within the FIM client in order to import these users. Please contact Microsoft support if you require assistance in getting this configured. We are also working on creating public documentation on how this configuration can be made.
   To find out if a users in SUN LDAP have custom structural object classes associated to them, please contact your SUN LDAP administrator. You can also use a tool such as LDP.exe to view the “object-class” LDAP property of the user.

I hope this has been a helpful post. Please leave your comments and issues that you are running into with profile synchronization between SharePoint and SUN LDAP, so I can update this post with more information.

Happy SharePointing!

 

 

 

Work Status: Must be a US Citizen as required by the Government contracts.

Location Springfield, VA
Duration: FULL TIME

Rate: 1099 or corp-to-corp (see bottom)***

Job summary - Developer - JAVA

The Developer works on complex application problems involving all phases of software development and maintenance. The developer analyzes requirements and develops detailed design specifications, develops block diagrams and logic flowcharts, and translates detailed designs to computer programs. The developer codes, tests, debugs, and refines new software or makes enhancements to existing programs to produce the required product. Developers must be able to participate in code reviews and provide constructive feedback to other developers.
Summary of functional responsibilities
• Design and develop software code within established specifications using object oriented applications, languages and third party tools.
• Design, revise and maintain flow charts setting out code functions and classes as assigned.
• Test software for bugs and clean the code. Write testing scripts to operate and test all elements of programs. Review results and make necessary changes. Attempt to replicate problems, determine causes and fix codes to eliminate recurrence of problems.
• Integrate codes with portions of software package developed by other team members.
• Ensure that all development efforts are integrated so that code is readily readable and maintainable by other team members.
• Research new methods, libraries and tool versions. Maintain current awareness.
• Ability to install own software.
• Assist in the mentorship of new developers and able to provide guidance on project standards.
Experience requirements
• Experience with independent programming, including responsibility for software development projects and/or demonstrated programming team leadership is preferred
• Security: Persons who fill this position will be required to receive a favorable background investigation prior to being assigned to fulfill all defined functional responsibilities. Persons with an existing BI are preferred.

Fundamental skills make up foundational knowledge that all developers must build on. Without this, the developer cannot function. Proficiency in the following is required:
• Core Java Programming Knowledge (qualified by objectives covered in the Sun Certified Java Programmer Certificate)
• Object Oriented Design Concepts
• Key Design Patterns (Gang of Four)
• SpringMVC Framework
• Java Sever Pages (JSP)
• Cascading Style Sheets (CSS)
• XML
• JDBC

The ability to acquire a majority of the skills listed below in a timely manner is integral to future career growth. Additional skills and applicable technologies utilized by most projects are as follows:
• WebSphere MQ Series
• Java Messaging Service (JMS)
• Spring Framework (Dependency Injection,
• XSD Schema Definition
• Hibernate
• JSR 168 Portlets

Minimum education requirement
• Bachelor’s degree in Computer Science or related field of study or an equivalent combination of education and experience from which comparable knowledge and skills may be acquired.
• Sun Certified Java Programmer Certificate (SCJP) is highly recommended.

*** PAY *** WHEN SENDING RESUMES PLEASE ALSO SEND US YOUR DESIRED HOURLY RATE for 1099 or corp to corp.

Today we'll combine the last two blog entries so that we can diff two databases and present the result visually to the user. The assumption is that you've followed everything described yesterday and that everything is up and running successfully. As a first next step, follow the blog entry from the day before that to set up your Maven-based NetBeans Platform application correctly for working with the Diff API. I.e., set the dependencies described in that blog entry, but don't create the TopComponent, since we'll handle that part differently.

At the end of this blog entry, you'll be able to diff two databases, as shown below, where you see the "Name" column of one database compared to the "Name" column of another database (where the databases have the same underlying structure):

The above could be pretty handy when (1) you have different versions of a database, e.g., version 1 and version 2, and you'd like to see the differences and/or (2) you have multiple different users making use of the same database structure (e.g., there's a customer application on top of a particular schema shared by multiple users and you'd like to see the difference between the database of User A and the database of User B). And a tool like the above is also a way to catch typos and other errors in databases.

As a further requirement, i.e., in addition to wanting to be able to diff databases, we also want to keep a history of diffs, which we'll then be able to reopen and compare with the current (or other diffs). To that end, we'll export the relevant parts of our databases to text files, compare those text files per project, and store the text files for later use (i.e., build up a history of diffs). Then, in the next blog entry, we'll learn how to display those text files in our application, enabling the user to choose, from the diff history, a particular diff for analysis.

So, let's get started!

Step 1: Create An Action To Handle Multiple Projects

With the above set up, i.e., you have the diff dependencies and you have done everything described yesterday, let's use the "New Action" wizard to create a conditionally enabled Action, with "User May Select Multiple Nodes" selected:

That will let the Action apply to the situation where TWO projects are selected, so that you can diff the underlying databases. In the "New Action" wizard, register the Action in the same place where you registered yesterday's Action, i.e., in the "CustomerSalesActions" category. We'll load the Action in the same way as the one from yesterday, i.e., into the context-sensitive list of Actions on the Node, rather than from a global menu item or global toolbar. Name the Action "DiffDatabasesAction", with "Diff Databases" as the display name.

When you complete the wizard, you'll have this Action, with new layer entries for the infrastructure to handle the underlying plumbing (make sure to make the Action asynchronous, via the layer attribute described yesterday). I.e., this is the Java class you now have, ready to be filled with the diff-related Java code:

import java.awt.event.ActionListener;
import java.awt.event.ActionEvent;
import java.util.List;
import org.netbeans.api.project.Project;

public final class DiffDatabasesAction implements ActionListener {

    private final List<Project> context;

    public DiffDatabasesAction(List<Project> context) {
        this.context = context;
    }

    public void actionPerformed(ActionEvent ev) {
        for (Project project : context) {
            // TODO use project
        }
    }
    
}

Note that we now have access to MULTIPLE projects, i.e., that's great, so now we have entry points for accessing the underlying databases (which are in fixed places within the projects, as explained yesterday), so that we can diff them.

Next, in the project's logical view, change the "getActions" to the following (or define it less constrictively, i.e., so that anything in our folder will be loaded):

@Override
public Action[] getActions(boolean arg0) {
    Action[] nodeActions = new Action[7];
    nodeActions[0] = Utilities.actionsForPath("Actions/CustomerSalesActions").get(0);
    nodeActions[1] = Utilities.actionsForPath("Actions/CustomerSalesActions").get(1);
    nodeActions[2] = CommonProjectActions.newFileAction();
    nodeActions[3] = CommonProjectActions.copyProjectAction();
    nodeActions[4] = CommonProjectActions.deleteProjectAction();
    nodeActions[5] = CommonProjectActions.setAsMainProjectAction();
    nodeActions[6] = CommonProjectActions.closeProjectAction();
    return nodeActions;
}

OK. Now everything is ready, except for all the actual functionality that we need. :-)

Step 2: Create FreeMarker Template Files

To diff the databases, we first need to export everything from them into files. Then we can diff the files. (Maybe this is not feasible for every imaginable scenario, so apologies if your trip down this road ends here.) We'll use FreeMarker as our templating engine (search this blog for other entries relating to this topic, since the NetBeans Platform has built in FreeMarker support).

To use the built-in FreeMarker support from the NetBeans Platform in our own application, we need to add the following dependency to the application module's POM:

<dependency>
    <groupId>org.netbeans.modules</groupId>
    <artifactId>org-netbeans-libs-freemarker</artifactId>
    <version>RELEASE691</version>
    <scope>runtime</scope>
</dependency>

Also, add this to the manifest of the functionality module:

OpenIDE-Module-Needs: javax.script.ScriptEngine.freemarker

OK, now FreeMarker is set up and ready for us to use it. In the folder where the "layer.xml" is found, create two text files, named "project0.txt" and "project1.txt". In the first file, write this content:

${content0}

In the second file, write this content:

${content1}

Later on, we will replace the variables above with strings that will contain the parts of the database that we'd like to diff.

Now go to the "layer.xml" file and register the above two templates as follows:

<folder name="CustomerSalesTemplates">
    <file name="project0.txt" url="project0.txt">
        <attr name="template" boolvalue="true"/>
        <attr name="javax.script.ScriptEngine" stringvalue="freemarker"/>
    </file>
    <file name="project1.txt" url="project1.txt">
        <attr name="template" boolvalue="true"/>
        <attr name="javax.script.ScriptEngine" stringvalue="freemarker"/>
    </file>
</folder>

From the above, you can see that we have a new folder in the filesystem, containing two text files, which we have registered as FreeMarker templates. That will enable the infrastructure to correctly convert the variables to the strings we will pass in later.

Step 3: Write Database Content To FreeMarker Template Files

Now fill out the "DiffDatabasesAction" class as follows, hopefully the comments inline will explain everything:

public final class DiffDatabasesAction implements ActionListener {

    private final List<Project> context;

    public DiffDatabasesAction(List<Project> context) {
        this.context = context;
    }

    @Override
    public void actionPerformed(ActionEvent ev) {
        
        try {

            //For the first project,
            //create a StringBuilder that will contain database content,
            //a HashMap to pass in to the FreeMarker template,
            //a FileObject for the registered template,
            //and a DataObject from the FileObject:
            StringBuilder sb0 = new StringBuilder();
            HashMap hashMap0 = new HashMap();
            FileObject template0 = FileUtil.getConfigFile("CustomerSalesTemplates/project0.txt");
            DataObject dTemplate0 = DataObject.find(template0);

            //For the second project,
            //do the same as the above:
            StringBuilder sb1 = new StringBuilder();
            HashMap hashMap1 = new HashMap();
            FileObject template1 = FileUtil.getConfigFile("CustomerSalesTemplates/project1.txt");
            DataObject dTemplate1 = DataObject.find(template1);

            //Start the progress bar:
            ProgressHandle handle = ProgressHandleFactory.createHandle("Comparing projects...");
            handle.start();
            
            //Assume we only want to compare two projects:
            if (context.size() == 2) {
                for (int i = 0; i < context.size(); i++) {
                    
                    //Get the current project:
                    Project project = context.get(i);
                    
                    //Create a db connection per project,
                    //using the utility class described yesterday:
                    DBProps config = new DBProps();
                    FileObject projDir = project.getProjectDirectory();
                    config.setDbHost(projDir.getPath() + "/.derby");
                    config.setDatabase("sample");
                    config.setUsername("app");
                    config.setPassword("app");
                    
                    //Create a new entity manager for the db:
                    EntityManagerFactory entityManagerFactory = Persistence.createEntityManagerFactory("com.customer_CustomerSalesDomain_nbm_1.0-SNAPSHOTPU", config.buildPersistenceProperties());
                    EntityManager entityManager = entityManagerFactory.createEntityManager();
                    
                    //Get all the Customer content from the current db,
                    //and write all the names into the StringBuilders:
                    Query query = entityManager.createQuery("Select c From Customer c");
                    List<Customer> resultList = query.getResultList();
                    for (Customer customer : resultList) {
                        if (i == 0) {
                            sb0.append(customer.getName()).append("\n");
                        } else if (i == 1) {
                            sb1.append(customer.getName()).append("\n");
                        }
                    }
                    
                }
                
                //Put the StringBuilders into the HashMap,
                //which will be passed to the template creator:
                hashMap0.put("content0", sb0.toString());
                hashMap1.put("content1", sb1.toString());
                
                //Get a time stamp, for the name of the folder
                //that will contain the two text files:
                String stamp = now().toString();
                
                //Create a "Diffs" folder, if it doesn't exist:
                FileObject diffs = FileUtil.getConfigFile("Diffs");
                if (diffs == null) {
                    FileUtil.getConfigRoot().createFolder("Diffs");
                }
                
                //Create a subfolder named after the timestamp,
                //and put the two files there:
                FileObject checkedDiffs = FileUtil.getConfigFile("Diffs").createFolder(stamp);
                DataFolder df = DataFolder.findFolder(checkedDiffs);
                DataObject dobj0 = dTemplate0.createFromTemplate(df, "project0", hashMap0);
                DataObject dobj1 = dTemplate1.createFromTemplate(df, "project1", hashMap1);
                FileObject createdFile0 = dobj0.getPrimaryFile();
                FileObject createdFile1 = dobj1.getPrimaryFile();
                
                //Prepare to pass the text files to the Diff Viewer:
                File textFile0 = FileUtil.toFile(createdFile0);
                File textFile1 = FileUtil.toFile(createdFile1);
                StreamSource project0 = StreamSource.createSource("name1",
                        "Project 1", "text/plain",
                        textFile0);
                StreamSource project1 = StreamSource.createSource("name2",
                        "Project 2", "text/plain",
                        textFile1);
                
                //Create the Diff Viewer:
                diff(project0, project1, stamp);
                
                //Stop the progress bar:
                handle.finish();
                
            }
            
        } catch (IOException ex) {
            Exceptions.printStackTrace(ex);
        }
    }

    //Create the Diff Viewer on the fly,
    //making sure that it won't be opened when the app restarts:
    public void diff(final StreamSource projectA, final StreamSource projectB, final String stamp) {
        SwingUtilities.invokeLater(new Runnable() {
            @Override
            public void run() {
                try {
                    DiffView view = Diff.getDefault().createDiff(projectA, projectB);
                    TopComponent tc = new TopComponent() {
                        @Override
                        public int getPersistenceType() {
                            return PERSISTENCE_NEVER;
                        }
                    };
                    tc.setDisplayName("Diff Viewer" + " -- " + stamp);
                    tc.setLayout(new BorderLayout());
                    tc.add(view.getComponent(), BorderLayout.CENTER);
                    tc.open();
                    tc.requestActive();
                } catch (IOException ex) {
                }
            }
        });
    }

    //Utility method for creating the timestamp:
    public static final String DATE_FORMAT_NOW = "yyyy-MM-dd HH:mm:ss";
    public String now() {
        Calendar cal = Calendar.getInstance();
        SimpleDateFormat sdf = new SimpleDateFormat(DATE_FORMAT_NOW);
        return sdf.format(cal.getTime());
    }

}

That's it, you're done. (Yes, lots of conversions between different types of objects, but unavoidable since different methods have different requirements and different functionality comes from different subsystems of the NetBeans Platform.) Run the application, open the projects, right-click on two of them, choose "Diff Databases", and you'll see something like this:

Next time, we'll read our "Diffs" folder, created in the user directory via the code above, and display the diff history so that the user can do interesting things with it (i.e., open a previous diff in the Diff Viewer and then compare it to a later diff).

This is the twenty-eighth in a series of blog posts I’m doing on the VS 2010 and .NET 4 release.

This blog post continues a series of posts I’m doing that cover the new improvements we made around web deployment.  In my first post in the deployment series I provided an overview of the new VS 2010 Web Deployment features.

In today’s post I’m going to be discussing Microsoft Web Deploy – which is a free server technology that enables a comprehensive publishing and deployment mechanism.  Web Deploy enables you to not only publish files – but also provision database schema/data, run database change scripts, set security ACLs, and more.  All of these deployment steps can be automated using Visual Studio.

This post will provide a step by step tutorial on how to install Web Deploy and enable it on a web-server.  I’ll then demonstrate how you can use Visual Studio 2010 to directly publish to servers using Web Deploy, as well as how you can use Visual Studio to create installation packages that you can use to automate the deployment of your applications.

Web Deploy – And Why Should You Care

Deploying a web application or site involves several steps. You typically need to:

• Copy files and folders
• Provision database schema and optionally populate database content
• Set appropriate read/write and security ACLs on files and folders
• Install and configure SSL certificates
• Deploy other misc dependencies (Event logs, GAC libraries, COM objects, etc)

Performing all of these steps manually is tiresome, slow, and dangerous (since manual deployment steps are error prone).  Enabling an automated process that allows you to quickly provision and deploy applications on a server helps reduce the chances of things going wrong, and can dramatically improve the cycle time it takes for you to get a change you make in an application into production. 

Microsoft Web Deploy

Web Deploy is a free technology built by my team that can improve web deployment. It is a deployment service that runs on a server, and enables you to either locally or remotely deploy applications onto it.  Web Deploy includes built-in support for creating virtual directories and applications within IIS, copying files and folders, provisioning databases (both SQL Server and MySQL), setting file-system ACLs and more (it even includes built-in providers for things like setting registry entries, registering COM components, etc).  In addition to having these built-in deployment features, it also supports a .NET based provider API that enables you to create and plug-in your own custom deployment actions – which allow you to customize it however you want.

The wire-protocol for Web Deploy is HTTPS – which means Web Deploy can be used through firewalls (enabling easy, secure, deployment to remote hosted servers).  Web Deploy also supports both admin and non-admin deployment scenarios.  The non-admin scenarios enable administrators to configure Web Deploy on a server, and then delegate a subset of deployment capabilities to developers who do not have admin accounts on the production server.  This enables a very secure and flexible deployment approach.  I’ll cover the configuration steps to enable delegated deployment scenarios in my next blog post in this series.

Installing Web Deploy on Windows Server 2008

Visual Studio 2010 (and Visual Web Developer 2010 Express) will by default install Web Deploy on your development machine as part of their setup.  This provides you with what you need to create Web Deployment setup packages within VS, as well as publish them to remote servers that have the Web Deploy service installed and enabled.

Below are the steps for how to install Web Deploy on a production server running IIS 7.x on Windows Server 2008 or Windows Server 2008 R2:

1) Download and run the Microsoft Web Platform Installer on your production server.

2) Click the “Web Platform” tab within the Web Platform Installer, then click the “Customize” link under the “Web Server” section.  Select the “Web Deployment Tool 1.1” feature as well as the “Management Service” feature:

3) Press the Install Button.  This will download and install the Web Deployment Package, and enable the IIS Management Service feature that is built-into Windows.

Enabling Deployments with Windows Server 2008

We now have Web Deploy installed – next we need to enable deployments with it.

Web Deploy supports deployments by administrators, as well as deployments by non-administrators (aka delegated deployments). In this blog post, I’m going to cover how to enable deployments using an account that has administrator permissions on the server.  I will cover how to enable delegated deployments by developers that do not have an administrator account in my next blog post in this series.

1) The first step to enable Web Deploy is to start the IIS Admin Tool (Start -> Run -> inetmgr.exe).  Then double-click the “Management Service Delegation” icon on the home-screen:

When you double-click the “Management Service Delegation” icon it will bring you to a page that looks like below.  In the Actions pane, click the “Edit Feature Settings…” link.  This will bring up the “Edit Feature Settings” dialog - check the “Allow administrators to bypass rules” option within this dialog box.  This will allow those with Administrator accounts on the server to bypass the delegation capabilities of the Web Management Service and perform actions using their administrator capabilities:

Then return back to the IIS Admin Tool home-screen and double click the “Management Service” icon:

 

When you double-click the “Management Service” icon it will bring you to a page that looks like below.  Click the “Enable Remote Connections” checkbox to enable remote deployments.  You can optionally choose which IP address and port the management service runs on – as well as what client IP addresses are allowed to connect with it.  You can tweak these settings to lock down who can access the deployment service.

When you are done click the “Start” link within the “Actions” tab to start the Web Management Service on the server:

The server is now setup for deployments using an administrator account with Web Deploy.

Important Tip: One thing to watch for is whether you have a firewall enabled on your server, or within the cluster where your server is hosted.  By default the Management Service runs using the HTTPS protocol on port 8172.  This port might be locked down by default depending on your hosting configuration.  If so you should either unlock it with your firewall/hoster – or pick a different port number that is allowed.  You can test to see whether the management service is available simply by opening up a browser and accessing it using a URL like: https://yourservername:8172/MsDeploy.axd – if you are prompted for a username/password then you know it is working, and there is no firewall blocking access to it.  If it times out then it is likely that a firewall is blocking it.

Enabling Web Deploy on a non-Windows Server 2008 Operating System

The steps above demonstrate how to enable the Web Deploy service on a Windows Server 2008 or Windows Server 2008 R2 operating system.  The IIS 7 Management Service is built-into these operating systems, and Web Deploy takes advantage of it.

If you try and follow the above steps on Windows 7, Windows Vista, or Windows Server 2003, though, you’ll notice that the IIS Management Service (and its icon within the IIS admin tool) isn’t available.  Instead you need to follow a slightly different set of steps to enable the Web Deploy service.

Enabling Web Deploy on Windows 7, Windows Vista, and Windows Server 2003

You can enable the Web Deploy publishing service on Windows 7, Windows Vista, and Windows Server 2003 using the below steps:

1) Open an elevated command prompt (meaning you right-click and launch it using the “run as administrator” command)

2) Type “net start msdepsvc” to start the “Web Deploy Agent Service”:

3) To confirm that the publishing service is working, change directory to the “c:\Program Files\IIS\Microsoft Web Deploy”, and then type the command “msdeploy –verb:dump –source:appHostConfig,computername=localhost”:

This should cause the local msdeploy client to connect to the publishing service you just enabled and dump out the current status of your web-servers ApplicationHost.Config file:

If a bunch of spew comes out then you know that everything is working and you have just enabled Web Deploy for publishing.  Using this approach won’t allow you to enable “delegated access” (which allows non-admin accounts to deploy – and which I’ll cover in more depth in my next blog post) – but will allow those with an account with admin permissions to deploy to the machine.

Using the “Publish Web” Dialog within Visual Studio

Now that we have our web server enabled with Web Deploy, let’s try deploying something to it within the Visual Studio IDE.

VS 2010 includes a new “Publish Web" feature that you can use to quickly deploy a web application to a remote server.  You can activate it by right-clicking on any ASP.NET Web Application Project (both Web Forms and MVC varieties), and then select the “Publish” context menu item:

Selecting this will bring up a “Publish Web” dialog which allows you to configure publish settings.  You can use this dialog to publish an ASP.NET application to a remote user using FTP/FTPS, Web Deploy, or FrontPage Server Extensions.

We’ll select the “Web Deploy” option from the drop-down, and then enter the publishing information of our Web Deploy server:

Note that you only have to fill these settings out once – you can then save them as a “Publish Profile” using the “Save/Rename/Delete” buttons at the top.  Publish profiles allow you to save multiple deployment/publishing options and quickly flip between them depending on what server you want to use.

A few notes about the various options you can specify:

Service URL:	This is the URL of the Web Deploy Management Service.  If you are using Windows Server 2008 the default format of the URL is https://mysite:8172/MsDeploy.axd Note the protocol is “https://” and the port should match whatever you specified when you enabled the IIS Management Service above.  If you are publishing against a Windows Server 2003, Windows 7, or Windows Vista machine then the default format of the URL is http://server-name/ (not https – since the security credentials are sent using built-in Windows authentication which is encrypted). You also don’t need to specify a port number with Windows Server 2003, Windows 7 or Windows Vista.
Site/Application:	This allows you to specify the site name (and optional application name) on the remote server that you want to install the application to.  You can publish your project as a site, in which case you might specify something like www.mysite.com as the sitename, or “Default Web Site” if you just want to publish at whatever the default root site name is.  Important: The site name you specify needs to correspond to the same site-name that shows up in the IIS admin tool.  So if you’ve registered the www.scottgu.com site using the friendly name “ScottGu’s Site” or just “scottgu.com” when you created it in IIS you need to make sure the site name you specify as a publish setting corresponds to the friendly name (so you’d specify “ScottGu’s site” instead of www.scottgu.com if that is what shows up in the IIS admin tool as the site name).  If you specify this incorrectly you’ll get an error that says the the remote agent “could not be contacted”. Alternatively you can publish to a sub-application location using a value like “www.mysite.com/myapplication” or “ScottGu Site/myapplication”.  If the /myapplication vdir and application doesn’t already exist, then the Web Deploy service will create it for you and then publish your project to it.  You can optionally indicate that you want this sub-path to be a virtual directory (the default) or an application by checking the “Mark as IIS Application on Destination” checkbox.
Credentials:	If you are publishing to a remote server that is not part of your windows domain, then you can specify your username/password in the publish dialog.  This username/password combination can either map to a Windows account on the remote server – or alternatively map to IIS usernames (which enable more flexible locked-down delegation options – which I’ll cover in my next blog post). Important: Unless you’ve installed a signed SSL certificate on your remote server, make sure you check the “Allow untrusted certificate” checkbox.  By default Web Deploy will install a cert for you to use that is unique (and unsigned) – and if you don’t check this checkbox your login will fail because VS won’t trust an unsigned cert.
Other Notes:	There is a checkbox provided that allows you to indicate whether you want to “Leave extra files on destination” or not.  If you uncheck this then the existing files in the site/application you are publishing to will be deleted if they don’t match what is currently in your VS project.

Once you have filled out the Publish settings, you can click the “Publish” button and it will connect to the remote Web Deploy service and deploy your application to the location you provided.  Your “Output” windows within Visual Studio will show output like below that explains how it was deployed and the actions that occurred with the publish step:

The above project is a basic web project with just a few files and directories within it.  Web Deploy will handle copying all of the file and setting appropriate ACLs on the remote server (for example: making the \app_data directory to be read/write).  In later blog posts in this series I’ll discuss how to enable database deployment as part of the Web Deploy process – as well as how to automatically switch web.config file settings (e.g. connection-strings) as part of it.

And with that we’ve published our site on a remote server.  To re-publish it again you can right-click on the project and once-again select the “Publish” command – which will bring up the publish dialog again (with the same settings as last time populated in it by default). Alternatively, you can enable the “Web One Click Publish” toolbar within VS 2010 – which enables you to quickly switch between saved publishing profiles (using a drop-down) and then click the “"Publish” button to the right of it to public the project to the target server (no dialog required):

Note that Web Deploy is smart enough to compare the remote server with your local project – and only copy the files it needs to.  So if you make a quick few changes to a large project, and then re-publish again – only those files that changed will be copied over.  The files that weren’t updated won’t need to be copied again.  This makes re-deploying/updating a site much faster – especially when there is a lot of static content and large images in the project.

Web Deploy also by default compresses files before it copies them to the Web Deploy service – which shrinks the wire-size and enables faster deployments.

Creating Deployment Packages from VS 2010

VS 2010 also supports a packaging option that enables you to package up your ASP.NET Web Application (together with its dependencies like web.config, databases, ACLs, etc) into a .zip based deployment package file that Web Deploy understands.  You can then hand-off the deployment package file to someone who can deploy it either via the IIS Admin Tool, or via a command-line/powershell script that installs it on a remote server using Web Deploy.

The deployment package you create can optionally expose application configuration settings that can be overridden (like directory locations, database connection-strings, etc).  When using the IIS7 Admin Tool, the install wizard will prompt the administrator for each setting to be customized – enabling you to provide a clean customization experience without having to write any custom code to-do so.  The settings can also be passed as arguments on the command-line when using a command-line or Powershell script to deploy the application via a remote Web Deploy service.

To create a web package within Visual Studio 2010, just right click on your ASP.NET Web Project in the solution explorer and select the “Build Deployment Package” menu item:

Selecting this option will cause VS 2010 to build a Web Deploy compatible deployment package.  You can configure where this package is created on disk within the project’s “Properties” dialog (select the Package/Publish Web tab).  Note that the disk location of the generated Web Deploy package is always displayed in the “Output” window build content (which makes it easier to find):

The generated package file is literally a .zip file that contains a Web Deploy manifest that enables you to easily deploy it on a remote server using either the IIS Admin Tool or a command-line.

Installing a Package Using the IIS Admin Tool

Once you have created the .zip deployment package file, you can use the IIS Admin Tool to install it.  Within the IIS Admin tool simply click the “Import Server or Site Package…” link on the “Actions” panel of the admin tool (these links are present if Web Deploy is installed):

This will launch a dialog that allows you to select the .zip package file.  When you do this the IIS Admin Tool will show the administrator the exact deployment steps that have been automated by the package:

If you continue through the wizard it will automate provisioning the application for you onto your server.

Deploying a Package from the Command-Line

Alternatively, you can use a command-shell to deploy the package onto a remote server.

To do this, open a command window, and navigate to the location of the deployment package .zip file on disk.  In addition to the .zip package file, you’ll find that VS also added a few additional files to the directory:

The ProjectName.deploy.cmd file provides a pre-built script that you can use to deploy the package onto a remote server. 

The ProjectName.SetParameters.xml file contains some deployment parameters that you can set (like site-name, connection-string locations, etc).  You can use Visual Studio’s project properties pane to set the default values for the parameters that are generated into this file.  Admins can then optionally edit/change them later if they want (avoiding the need to explicitly specify them as command-line parameters). Make sure to open/view this file to see what the current defaults in it are.  In particular, check to make sure that the site name/application name is where you want the application deployed.

To install the package on a remote Web Deploy server, run the deployment script with the following parameters:

ProjectName.deploy.cmd /y /M:https://WebDeployUrl:8172/MsDeploy.axd /u:username /p:password –allowUntrusted /A:basic

A couple of quick notes on the various parameters above:

/y	Indicates that Web Deploy should deploy the application onto the remote server.  Alternatively instead of /y you can specify /t to perform a trial install that will simulate deployment and help you verify everything is ready (without actually provisioning it).
/M: parameter	Specifies the Web Deploy publishing endpoint of the server you want to deploy the package onto.  It should match whatever publishing service URL you setup (and will be the same as the “Service URL” parameter in the Publish Web dialog). It should be an “https” based URL if you are publishing to a Windows Server 2008 or 2008 R2 server.
-allowUntrusted	Required if the SSL certificate on the remote Web Deploy server is unsigned (which it isn’t by default)
/A:basic	Required if the remote server is not using Windows Authentication to identify the user (it instead specifies that you will use http basic auth over SSL)

When you run the command it will deploy the package to the remote server, perform all of the deployment steps (including things like provisioning databases, setting file ACLs, etc), and output the status back to the command-prompt as it makes progress.

Automating Deployment from a Build Server or Continuous Integration Server

In addition to having a developer/administrator kick off deployment explicitly, you can alternatively automate it to happen as part of a continuous integration process or as part of a build server.  The commands necessary to automate creating a Web Deploy .zip package are available as MSBuild tasks.  You can use them to either create a deployment package as part of the build process – or optionally have an automated build also deploy the app as well.

I’ll be covering how to enable these automated build scenarios in a future blog post.

Deploying Across a Web Farm

Web Deploy can be used together with the Microsoft Web Farm Framework to enable automated deployment across a web-farm.  You can install and configure Web Deploy on a primary server in a Web Farm Framework cluster – and the secondary servers within the web farm cluster will then monitor and clone any applications you deploy with it.  This allows you to use the exact same steps as above to deploy sites and applications across any number of machines within a web farm in an automated way.

Read my previous blog post about the Microsoft Web Farm Framework to learn more about how to enable this.

Summary

Web Deploy provides a powerful and flexible way to automate the deployment of ASP.NET Web Applications to a remote server.  Web Deploy enables you to not only publish files – but also provision database schema/data, run database change scripts, set security ACLs, and more. 

You can use Visual Studio 2010 to directly publish to web servers that have Web Deploy enabled, or create deployment package files that can be installed either via an admin tool or the command-line.  You can also integrate packaging and deployment as part of a build server or continuous integration process to better enable a continuous delivery model.

I’ll cover more about how to enable Web Deploy with delegated security scenarios in my next blog post.  I’ll then do blog posts that cover how to modify/customize web.config files as part of deployment, how to deploy databases as part of a Web Deploy process, and how to integrate all of this as part of an automated build process.

Hope this helps,

Scott

P.S. In addition to blogging, I am also now using Twitter for quick updates and to share links. Follow me at: twitter.com/scottgu

Nowadays, I tend to use a typed link leading to a form, rather than a heavily typed link alone (I’ll explain what I mean by heavily typed link shortly), to advertise unsafe operations and/or requests that require an entity body. Here’s an example of a typed link:

//Request
GET /shop HTTP/1.1
Host: restbucks.com
Accept: application/vnd.restbucks+xml

//Response
HTTP/1.1 200 OK
Date: Mon, 26 Jul 2010 10:00:00 GMT
Cache-Control: public, max-age=86400
Content-Type: application/vnd.restbucks+xml
Content-Length: ...

<shop xmlns="http://schemas.restbucks.com/shop">
  <link rel="http://relations.restbucks.com/rfq"
        href="http://restbucks.com/request-for-quote"
        type="application/vnd.restbucks+xml"/>
</shop>

The link here is typed with the link relation value http://relations.restbucks.com/rfq, which indicates that the link points to a resource where a request for a quote can be submitted. Following the link, the client retrieves a form:

//Request
GET /request-for-quote HTTP/1.1
Host: restbucks.com
Accept: application/vnd.restbucks+xml

//Response
HTTP/1.1 200 OK
Date: Mon, 26 Jul 2010 10:00:05 GMT
Cache-Control: public, max-age=86400
Content-Type: application/vnd.restbucks+xml
Content-Length: ...

<model xmlns="http://www.w3.org/2002/xforms"
  schema="http://schemas.restbucks.com/rfq.xsd">
  <submission
    resource="http://restbucks.com/quotes" 
    method="post" 
    mediatype="application/vnd.restbucks+xml"/>
</model>

The response entity body comprises an XForms form model. (Our custom media type definition for application/vnd.restbucks+xml says that one of the things a client can expect to receive in a response is an XForms form model.) The form’s <submission> element includes several pieces of control data: it indicates which verb to use when submitting the form (POST), where to submit the form (http://restbucks.com/quotes), and which content or media type to use when submitting the form (application/vnd.restbucks+xml). Because the definition of our custom media type includes more than one XML schema (much as the Atom specification defines two schemas, one for feeds and one for entries), the form’s control data needs to further clarify what /quotes expects to receive in a POST request. To clarify which schema the POST request body should adhere to, the <model> element’s schema attribute references the rfq.xsd schema.

Here, then, we have all the information a client needs to construct and submit a valid request to the /quotes resource. And all without any fields for the client to fill out.

Heavily typed links

A heavily typed link is one where the link relation describes not only the relationship to the linked resource, but also the HTTP idioms – the control data – necessary to manipulate that resource.

Using a heavily typed link our shop could link directly to the /quotes resource, instead of to a form:

<shop xmlns="http://schemas.restbucks.com/shop">
  <link rel="http://relations.restbucks.com/quotes"
        href="http://restbucks.com/quotes"
        type="application/vnd.restbucks+xml"/>
</shop>

Here, the definition of the link relation http://relations.restbucks.com/quotes might be something like: “Indicates a collection of quotes. To request a quote, POST a <request-for-quote> with a Content-Type header of application/vnd.restbucks+xml to the linked resource.”

That’s a perfectly respectable way of using links and link relations; in fact, it’s the strategy we employ in REST in Practice. But it can have its downsides. Most importantly, it can increase the coupling between the client and any resources that employ link relations.

Make no mistake: link relation semantics comprise out-of-band knowledge. There’s no magic here: link relations introduce a degree of coupling between a client and any server-governed resources that adopt them. The trick is to keep this coupling as low as possible. By putting control data in the link relation definition, we perhaps introduce more coupling than is strictly necessary.

Out-of-band data is less visible, and more difficult and more costly to change, than data that is inlined in the message. Whilst the control data may not change all that often, changes can and do sometimes happen; inlining the data allows these changes to be propagated to clients sooner rather than later. Control data produced at the time the response is generated is generally more recent than control data defined through some out-of-band mechanism.

Using lightly typed links helps separate semantics from control data. A “light” link relation indicates what the linked resource means in the context of the current representation – that’s all. This helps mitigate against a second, somewhat more subtle, downside of adding control data to link relations: the tendency to introduce action semantics. It’s no great step to shorten the link relation value above to http://relations.restbucks.com/quote, and to rewrite its semantic to read “Indicates an opportunity to request a quote by POSTing a <request-for-quote> with a Content-Type header of application/vnd.restbucks+xml to the linked resource.” At this point, our link has effectively become an operation. The typed-link-to-form strategy helps us concentrate on describing what a linked resource is, rather than what a link does. Link relations do not necessarily imply action semantics, but they can very easily be made to do so.

(Note: when adding links to representations, I still prefer to use a <link rel="..." href="..."> construct, or something similar, rather than <order-form href="...">. The reason for this is that elements such as <link> separate link syntax from semantic context, as explained here; and this is a good thing, because what a link ought look like – its syntax – changes far less than what a link might mean in a particular context. By separating these concerns, we make it easier to evolve a distributed application.)

Interestingly, on the wire, the result of submitting an XForm form looks exactly the same as if we’d simply POSTed a <request-for-quote> directly to /quotes:

//Request
POST /quotes HTTP/1.1
Host: restbucks.com
Content-Type: application/vnd.restbucks+xml
Content-Length: ...

<request-for-quote xmlns="http://schemas.restbucks.com/rfq">
  <items>
    <item>
      <description>Costa Rica Tarrazu</description>
      <amount>250g</amount>
    </item>
    <item>
      <description>Guatemala Elephant Beans</description>
      <amount>250g</amount>
    </item>
  </items>
</request-for-quote>

Knowing this, we could always add both a lightly typed link to a form and a heavily typed link to our shop representation:

<shop xmlns="http://schemas.restbucks.com/shop">
  <link rel="http://relations.restbucks.com/rfq"
        href="http://restbucks.com/request-for-quote"
        type="application/vnd.restbucks+xml"/>
  <link rel="http://relations.restbucks.com/quotes"
        href="http://restbucks.com/quotes"
        type="application/vnd.restbucks+xml"/>
</shop>

Two paths to the same result. I don’t recommend doing this: I include simply to highlight how a linked form achieves the exact same result, but with the added beenfit of having inlined control data.

Pre-filled forms/self-describing requests

Here’s another example of using a typed link to a form:

//Request
GET /quotes/1234
Host: restbucks.com
Accept: application/vnd.restbucks+xml

//Response
HTTP/1.1 200 OK
Cache-Control: public
Date: Mon, 26 Jul 2010 10:01:00 GMT
Expires: Mon, 02 Aug 2010 10:01:00 GMT
Content-Type: application/vnd.restbucks+xml
Content-Length: ...

<quote xmlns="http://schemas.restbucks.com/quote">
  <items>
    <item>
      <description>Costa Rica Tarrazu</description>
      <amount>250g</amount>
      <price currency="GBP">4.40</price>
    </item>
    <item>
      <description>Guatemala Elephant Beans</description>
      <amount>250g</amount>
      <price currency="GBP">5.30</price>
    </item>
  </items>
  <link rel="http://relations.restbucks.com/order-form"
        href="http://restbucks.com/order-forms/1234"
        type="application/vnd.restbucks+xml"/>
</quote>

The link relation http://relations.restbucks.com/order-form indicates that the linked resource is something that allows an order to be submitted. Following this link, the client retrieves an order form:

//Request
GET /order-forms/1234
Host: restbucks.com
Accept: application/vnd.restbucks+xml

//Response
HTTP/1.1 200 OK
Cache-Control: public
Date: Mon, 26 Jul 2010 10:01:05 GMT
Expires: Mon, 02 Aug 2010 10:01:00 GMT
Content-Type: application/vnd.restbucks+xml
Content-Length: ...
Content-Location: http://restbucks.com/quotes/1234

<model xmlns="http://www.w3.org/2002/xforms">
  <instance>
    <quote xmlns="http://schemas.restbucks.com/quote">
      <items>
        <item>
          <description>Costa Rica Tarrazu</description>
          <amount>250g</amount>
          <price currency="GBP">4.40</price>
        </item>
        <item>
          <description>Guatemala Elephant Beans</description>
          <amount>250g</amount>
          <price currency="GBP">5.30</price>
        </item>
      </items>
      <link rel="self"
            href="http://restbucks.com/quotes/1234"
            type="application/vnd.restbucks+xml"/>
    </quote>
  </instance>
  <submission
    resource="http://restbucks.com/orders" 
    method="post" 
    mediatype="application/vnd.restbucks+xml"/>
</model>

Once again, I’ve used an XForms form model, but this time I’ve pre-filled it with an <instance> element. Even so, there are no form fields to fill in; all the client needs to do is operate the form according to the inlined control data.

The interesting thing here is that /order-forms/1234 simply returns a different representation of the quote resource identified by /quotes/1234 (the response indicates as much in its Content-Location header). By supplying a forms-based representation of the quote, we inline all the information necessary to submit an order to an order processing engine. The client doesn’t need to compose an entity body; it simply needs to operate the form according to its control data. This results in a self-describing message being sent to /orders.

(In a real-world application I’d likely include a signature in the form body. This signature would guarantee that the client hasn’t tampered with the form contents prior to submitting the form to the order processing engine. The effectiveness of the signature depends on an out-of-band trust relationship having been established between the quoting engine and the order processing engine.)

The result of first following the link to the form, and then submitting the form, is to transition the overall state of the distributed application from Quote Requested to Order Placed, as illustrated in the following diagram:

Every request-response pair transforms application state. Retrieving the form enriches the client’s understanding of the current application state; that is, it opens up new opportunities for interacting with other resources. POSTing the form causes the application state to transition from Quote Requested to Order Placed.

The fact that the overall state of the application has changed is of no consequence to the server resources involved; the application state model is nowhere baked into the server resources. As far as the quote resource is concerned, it has simply been asked to surface a forms-based representation of itself. As far as the orders resource is concerned, it has simply created a new, subordinate order resource. This change in application state is, however, important to the client.

Summary

Understand the tradeoffs between inlining and putting control data in an out-of-band mechanism. Use the right controls for the job; understand the many different hypermedia capabilities at your disposal. The best resource for this is Mike Amundsen’s in-depth study of the hypermedia capabilities of many different kinds of hypermedia control. Recently, Andrew Wahbe started examining the need for machine-to-machine hypermedia, and the differences between controls for machines and controls for humans. Watch his blog for further discussion.

My understanding of hypermedia controls has been heavily influenced by my experience of the human web, where links and forms predominate. But when we talk about forms in a machine-to-machine context, it’s not the form field elements that are of interest, it’s the control data elements. These control data elements help program the client on the fly. The term “form” as it applies in a machine-to-machine context is likely an inappropriate metaphor; nonetheless, it does emphasize the fact that unsafe requests – and requests that have an entity body – require different hypermedia capabilities from simple GETs.

Because in the past I’ve tended to think form fields are redundant in machine-to-machine scenarios, I’ve avoided using forms at all, and have instead overloaded link relations with control data. But link relations are not a “get out of jail free” card. In overloading link relations, we add unnecessary coupling.

Coupling, of course, is not an all-or-nothing affair. There are degrees of coupling. We choose to accept some coupling because of the benefits it brings. But that doesn’t mean we should accept more coupling than is necessary; doing so can only inhibit our ability to evolve a distributed application.

 

1. Designing the Layers of a Solution (22%)

Design a loosely coupled layered architecture.

·         Separation of concerns including presentation

o   Business logic

o   Data

·         Minimizing logical dependencies

·         Deciding how layers connect

o   That is: content-based versus context-based filtered routing

Design service interaction.

·         Service granularity

o   Cohesiveness

·         Interface granularity (responsibilities of an operation)

·         Versioning, data and service contracts (using a message contract rather than a data contract)

·         Hosting and protocol

·         Managing data integrity

o   Re-validating data across trust boundaries

·         Evaluating use of RESTful interface

o   URI/resource structure

·         Choosing a message exchange pattern

·         Choosing synchronous vs. asynchronous operation invocation

·         Deciding whether to use custom SOAP headers

This objective does not include:

·         Interacting with existing/external systems

Design the security implementation.

·         Protecting data during transmission (encryption/hashing algorithm)

·         Authentication (client/proxy credential) and authorization

o   Authentication

§  Client/Proxy credential

o   Authorization

§  Group

§  Built-in or custom role provider

§  Claims Federated security

·         Designing for least privilege

o   Impersonation

o   Delegation

·         Planning for User Access Control (UAC) environments

·         Auditing

Design for interoperability with external systems.

·         Choosing an appropriate strategy for communicating with

o   COM components

o   Mainframe services

o   Web services

Design for optimal processing.

This objective may include but is not limited to:

·         Parallel processing

·         Asynchronous processing

·         Service bus

·         Gateway processes

·         Scalability (scale out vs. scale up)

§  Designing tiers to minimize latency

§  Batch retrieval

§  Multiple small calls

Design for globalization and localization.

This objective may include but is not limited to:

Multi-locale services

·         Designing for:

o   Time zone

o   Sorting

o   UI considerations

Database design considerations

 

2. Designing the Presentation Layer (21%)

Choose the appropriate Windows Client technology.

·         Choosing between Windows Forms, WPF, or a combination

·         Choosing an appropriate presentation pattern:

o   Model View Presenter [MVP]

o   Model View/View Model [MV-VM])

·         Identifying areas for possible migration/interoperability from Windows Forms to WPF

Design the UI layout and structure.

·         Evaluating the conceptual design

·         Deciding how the UI will be composed (e.g., static vs. dynamic screen)

·         Designing for the inheritance and re-use of visual elements (e.g., styles, resources)

·         Accessibility consideration

·         Deciding when custom controls are needed

Design application workflow.

·         User navigation

·         Designing wizards

·         Modal vs. non-modal

·         Dependencies among UI elements

·         Designing for input types based on environment and audience (kiosk, very large display, small display, indoors and outdoors)

Design data presentation and input.

·         Designing data validation

·         Designing a data-binding strategy

·         Designing a reporting strategy

·         Choosing media services (audio, video, images, animation)

·         Managing data shared between forms

Design presentation behavior.

·         Determining which behaviors will be implemented and how

·         Drag and drop functionality

Design for UI responsiveness.

·         Offloading operations from UI thread and reporting of progress

·         Avoiding unnecessary screen refresh

·         Media buffering; client vs. server side sorting and filtering of data

·         Addressing UI memory issues

 

3. Designing the Data Access Layer (21%)

Choose the appropriate data access strategy.

This objective may include but is not limited to: 

·         Choosing the appropriate data access technology (Entity Framework, LINQ to SQL, Microsoft ADO.NET)

·         Supporting data sources such as XML data, flat files, and relational databases

Design the data object model.

This objective may include but is not limited to:

·         Mapping to persistent storage (mapping to tables, XML files)

·         Abstracting from the service layer (encapsulating underlying schema details)

·         Designing a schema change management strategy

Design data caching.

This objective may include but is not limited to:

·         Managing data cache (lifetime, targets, size, scope)

·         Managing data state (change notification, cache invalidation/synchronization)

Design offline storage and data synchronization.

This objective may include but is not limited to:

·         Managing offline data

·         Mapping a data store to:

o   Local cache

o   Designing synchronization

·         Analyzing target data environment (e.g., Microsoft SQL Server, SQL Express, workstation capabilities, OS, bandwidth, reliability)

Design for a concurrent multi-user environment.

This objective may include but is not limited to:

·         Planning for concurrency and collision avoidance

·         Optimistic vs. pessimistic locking

·         Cross-tier distributed transactions

Analyze data services for optimization.

This objective may include but is not limited to:

·         Object relational mapping (ORM) performance

·         Optimizing roundtrips

·         Lazy vs eager loading

·         Caching of frequently used data

 

4. Planning a Solution Deployment (17%)

Define a client deployment strategy.

This objective may include but is not limited to:

·         Recommending an installation method (Xcopy, ClickOnce, MSI, third party)

·         Identifying prerequisites (target framework and bootstrap requirements)

·         Deploying COM components

Plan a database deployment.

This objective may include but is not limited to:

·         Existing or shared instance

·         Remote server

·         Embedded database

·         Deploying new objects (such as tables, stored procedures, and views) to a new or existing database; recognizing database security concerns (such as shared instances and access)

·         Remote vs. local database

This objective does not include:

·         DLL deployment

·         Shared GAC deployment

Design a solution update strategy.

This objective may include but is not limited to:

·         Preserving shared components

·         Data integrity

·         User customizations

·         Designing an update delivery method (e.g., automated update detection from the client)

·         Version mismatch (both local binaries and service interfaces)

As many of you will have seen, we’re curating a series of Google Maps showing key scientific locations in cities around the world (so far: London, Münster, San Francisco). We’re very keen to add further locations, and are hoping a few of you will relish the idea of putting such a map together for your own city (or region). If so, here’s a recipe for creating your map.Creating a map# Before you start, email Matt (i.am.mattbrown – at – gmail.com) to check no one else has the city covered.# You’ll need a Google account. If you haven’t got one, create one. If you have, log in and go to Google Maps.# Once the maps page loads, click ‘My maps’ up in the light-blue band near the top.# Click ‘Create new map’ and give it a name (e.g. ‘science in XYZ’) and brief description.# You should now see three editing tools in the top left corner of the map# The hand shape is to drag the map around. # The pin shape lets you place a pin in the map. Give that a try now, by clicking the pin, then clicking on a scientific location in your region. Once it’s in place, a dialog box will appear asking you for a description. You’ll also then see a ‘rich text’ link in the dialog box – this allows you to make text bold, italic, insert pics, etc. You can also change the style of the pin by clicking the pin icon in the dialog box.# The zigzag line lets you draw lines on the map of different thickness and colour. This might not be of much use for current purposes, but you might consider it for scientific ‘trails’ or walks. This tool (using the dropdown arrow) also lets you draw a shape like a square or oblong. It will fill the shape with a colour of your choosing. Again, perhaps best avoided for current purposes.# The map will autosave after every few minutes. But if you want to take a break at any point, just hit ‘save’ over on the left to make sure.# Once you’re happy, email Matt with a link to let him know the map is ready. To get the link, click the ‘Link’ text on the extreme right and copy the code from the top box.Notes on contentWhat to include in your map will depend on the peculiarities of your city or region. For example, if there are hundreds of biotechs you may wish to leave them off, or only include the top 10. Likewise, if there are a handful of historic sites pertinent to science then include them all. If there are hundreds, then cherry-pick the most interesting/important. You can use different coloured pins for different types of institution. The suggested schema would follow that of the London map:Red = museumGreen = learned societyTurquoise = university/collegePurple = publishing/libraryBlue = historic site(biotechs were left off this map, but could be given another colour)After creation of your map, we’ll publish it on Nature Network’s Schemes and Memes blog. You should keep an eye out for comments and update the map in light of any reasonable suggestions. The idea is that the maps are ‘community generated’ rather than initially authoritative and comprehensive.And that’s it. For anything more complex, get in touch with Matt.

A developer’s major concern is: How do I do application & database upgrades with minimal downtime? How do I synchronize between a DB’s version upgrade and an application’s version upgrade? I will break down the discussion into types of database refactoring operations, and I will limit to single table refactoring. The discussion will try to understand the need for refactoring and will dictate the steps towards a successful upgrade. Reader prerequisites I will assume MySQL to be the underlying database. To take a major component out of the equation: we may need to deal with very large tables, for which an ALTER command may take long hours. I will assume familiarity with Master-Master (Active-Passive) replication, with possible use of MMM for MySQL. When I describe “Failover from M1 to M2“, I mean “Make the ALTER changes on M2 (passive), then switch your application from M1 to M2 (change of IPs, VIP, etc.), promoting M2 to active position, then apply same changes on M1 (now passive) or completely rebuild it”. Phew, a one sentence description of M-M usage… I also assume the reader’s understanding that a table’s schema can be different on master & slave, which is the basis for the “use replication for refactoring” trick. But it cannot be too different, or, to be precise, the two schemata must both support the ongoing queries for the table. A full discussion of the above is beyond the scope of this post. Types of refactoring needs As I limit this discussion to single table refactoring,we can look at major refactoring operations and their impact on application & upgrades. We will discuss ADD/DROP COLUMN, ADD/DROP INDEX, ADD/DROP UNIQUE INDEX, ADD/DROP FOREIGN KEY, ADD/DROP TABLE. We will assume the database and application are both in Version #1 (V1), and need to be upgraded to V2 or greater. ADD INDEX Starting with the easier actions. Why would you add an index? Either: There is some existing query which can be optimized by the new query Or there is some new functionality which issues a query for which the new index is required. Adding an index is an easy action in that the table’s data does not really change. In case #1, all you need to do is to add the new index (if the table is large, fail over from M1 to M2). There is no application upgrade, so all that happens is that the database upgrades V1 -> V2. In case #2, the database must be prepared with new schema before the new functionality/query is introduced (since it depends on the existence of the index). The steps, therefore, are: DB: V1 -> V2 (possibly failover from M1 to M2) (Sometime later) App: V1 -> V2. Application will issue queries which utilize the new index. The application does not have to be upgraded at the same instant the DB gets upgraded. In fact, we’ll see that this is a typical scenario: we can separate upgrades into smaller steps, which allow for time lapse. One could work out steps 1 & 2 together, but that would take an extra effort. ADD COLUMN This must be one of the most common table schema upgrades: a new property is needed on the application side. It must be supported by the database. Perhaps a new field in some Java Object, with Hibernate mapping that field onto a new column. Or maybe the new column is there for purpose of de-normalization. This is also a more complicated task. Let’s look at the required steps: DB: V1 -> V2 (possibly failover from M1 to M2), change is ADD COLUMN. App: V1 -> V2. Change is: provide column value for newly INSERTed rows. If needed, retroactively update column values for all pre-existing rows. App: V2 -> V3. Application begins to use (read, SELECT) new column. The above procedure assumes that the new column must have some calculated value. A 10-million rows table must now be updated, to have the correct values filled in. So we ask of the application to start filling in data for new rows, which makes the invalid row set static. We can just take a “from row” and a “to row” and fill in the missing column’s value for those rows. Only when all rows contain valid values can we let the application start using that row. This makes for two application upgrades. If you’re content with just a static DEFAULT value, then step 3 can be skipped, and step 4 can be merged with step 2. ADD UNIQUE INDEX This is an altogether different case than the normal ADD INDEX, even though they may seem similar. And the case is particularly different when using Master-Slave failover for rebuilding the table. Consider the case where we add a UNIQUE INDEX on a slave. Some INSERT query executes on the master, successfully, and is logged to the binary log. The slave picks it up, tries to execute it, to find that it fails on a DUPLICATE KEY error. The UNIQUE INDEX is a constraint, and it makes the slave more constrained than the master. This is a delicate situation. Here how to (mostly) work it out: App: V1 -> V2. Change INSERT queries on relevant table to INSERT IGNORE or REPLACE queries, whichever is more appropriate. DB: V1 -> V2 (possibly failover from M1 to M2), change is ADD UNIQUE KEY (and while at it, a tip: are you aware of ALTER IGNORE TABLE?) The change of query ensures that the query will succeed on the slave (either by silently doing nothing or by actually replacing content). It also means that the slave can now have different data than the master. Of course, it you trust your application to never INSERT duplicates, you can sleep better. We do not handle UPDATE statements here. ADD CONSTRAINT FOREIGN KEY As with ADD UNIQUE INDEX, there is a new constraint here. A slave becomes more constrained than the master. But we now have to make sure INSERT, UPDATE and DELETE statements all go peacefully (well, it also depends on the type of ON DELETE and ON UPDATE property of the FK). The steps would be: DB: V1 -> V2 (possibly failover from M1 to M2), change is ADD CONSTRAINT FOREIGN KEY. And then cross your fingers or have trust in your application. If the table is small enough, one does not have to use replication to do the refactoring, and life is simpler. Just execute the ALTER on the active master, and continue with your life. CREATE TABLE This is a simple case, since the table is new. The steps are: DB: V1 -> V2 (no need to use slaves here) App: V1 -> V2. Application will start using new table. Conslusion Having such steps formalized help with development management and database management. It makes clear what is expected of the application, and what is expected of the database. The breaking down of these operations into sequential steps allows us to work more slowly; make preparation work; work within our own working hours; get a chance to see the family. In this post we took a look at “creation” refactoring changes. New columns, new keys, new constraints. In the next part of this article, we’ll discuss DROP operations.

The previous three posts in this series introduced all the components necessary to describe and model office space allocation problems. Here’s the payoff: a C# program that uses the components to solve problems. The important classes are:

• OfficeSpaceData. This class stores information about the entities, rooms, and constraints (e.g. “two entities should share a room”) that make up a problem instance.
• OfficeSpaceModelSimple. This class encapsulates a Solver Foundation model for the office space allocation problem. Callers invoke methods to describe constraints, and then an Allocate() method to solve the model.
• Solution. This class represents a solution for a problem instance, in particular the room assignment for each entity in the model. A Solution instance is returned by the OfficeSpaceModelSimple.Allocate() method.

Let’s modify the test program from this post to use all three components. The complete test code is below. Note the following:

• The Main method passes the XML filename, schema filename, and time limit to Run(). (Here is a link to some sample problems.)
• Run() does the following:
  • Creates a new OfficeSpaceData which reads the data,
  • Establishes the penalties for constraint violation,
  • Creates a new OfficeSpaceModelSimple, passing in the penalty weights,
  • Calls the Fill() method (see below),
  • Solves the model,
  • Reports the results (see below).
• Fill() scans through the constraints in the OfficeSpaceData object and calls the right methods on OfficeSpaceModelSimple. Remember that constraints have an “IsHard” property that specifies whether a constraint is strictly enforced, but that OfficeSpaceModelSimple currently ignores this setting and treats all constraints as “hard”.
• The Report() method prints out the results in an easy-to-understand format. The assignment is read from the Solution object. We also manually calculate the wastage and overuse penalties from the assignment. Finally, we call an Evaluate() method that loops over all the constraints and uses the SolutionEvaluator class to calculate the cost of the solution. This calculation is redundant it is supposed to be exactly the same as the goal in our model! I have included this redundant calculation on purpose to check that the model is indeed solving the right problem.

using System;    using System.Collections.Generic;  using System.IO;  using System.Linq;    using System.Text;      namespace Samples.OfficeSpace {    public class Program {        private bool printAssignment = true;      private bool includeDefaultConstraints = true;        static void Main(string[] args) {        if (args.Length == 0 || !args[0].EndsWith(".xml", StringComparison.InvariantCultureIgnoreCase)) {          Console.WriteLine("Please specify the path to an XML file.");          return;        }          string fileName = args[0];        if (!File.Exists(fileName)) {          Console.WriteLine("Could not find input file '{0}'.", fileName);          return;        }          string schemaFileName = fileName.ToLowerInvariant().Replace(".xml", ".schema.xml");          int seconds = args.Length > 1 ? Int32.Parse(args[1]) : 10;        Program p = new Program();        p.Run(fileName, schemaFileName, TimeSpan.FromSeconds(seconds));      }        private void Run(string fileName, string schemaFilename, TimeSpan limit) {        Console.WriteLine("FILE: {0}", fileName);          OfficeSpaceData data = new OfficeSpaceData();        data.ReadFromFile(fileName, schemaFilename);          Dictionary<ConstraintType, double> penalties = GetPenaltyWeights();        OfficeSpaceModelSimple model = new OfficeSpaceModelSimple(penalties);        model.SolveTimeLimit = limit;        string modelName = fileName.Substring(0, fileName.IndexOf('.'));        Fill(modelName, data, model);        Solution solution = model.Allocate();        Report(data, penalties, solution);      }
      private static Dictionary<ConstraintType, double> GetPenaltyWeights() {        // page 5        Dictionary<ConstraintType, double> penalties = new Dictionary<ConstraintType, double>();        penalties[ConstraintType.Adjacent] = 10;        penalties[ConstraintType.AllocateTo] = 20;        penalties[ConstraintType.AwayFrom] = 10;        penalties[ConstraintType.NoSharing] = 50;        penalties[ConstraintType.SameRoom] = 10;        penalties[ConstraintType.GroupBy] = 11.18;        penalties[ConstraintType.Wastage] = 1;        penalties[ConstraintType.Overuse] = 2;          return penalties;      }        private void Fill(string modelName, OfficeSpaceData data, OfficeSpaceModelSimple model) {        model.InitializeModel(modelName);        model.SetSizeBinding(data.Rooms, "Size", "ID");        model.SetSpaceBinding(data.Entities, "Space", "Name");          if (includeDefaultConstraints) {          model.Wastage();          model.Overuse();        }          Console.WriteLine("PROCESSING {0} CONSTRAINTS", data.Constraints.Count());        string e1, e2, room;        foreach (var constraint in data.Constraints) {          switch (constraint.Type) {          case ConstraintType.NoSharing:            if (TryGetSubjectEntity(data, constraint, out e1)) {              model.NoSharing(constraint.IsHard, e1, data.EntityCount);            }            break;            case ConstraintType.AllocateTo:            if (TryGetSubjectEntity(data, constraint, out e1) && TryGetTargetRoom(data, constraint, out room)) {              model.AllocateTo(constraint.IsHard, e1, room);            }            break;            case ConstraintType.GroupBy:            if (TryGetSubjectEntity(data, constraint, out e1) && TryGetTargetEntity(data, constraint, out e2)) {              model.GroupBy(constraint.IsHard, data.Rooms, e1, e2);            }            break;            case ConstraintType.SameRoom:            if (TryGetSubjectEntity(data, constraint, out e1) && TryGetTargetEntity(data, constraint, out e2)) {              model.SameRoom(constraint.IsHard, e1, e2);            }            break;            case ConstraintType.Adjacent:            if (TryGetSubjectEntity(data, constraint, out e1) && TryGetTargetEntity(data, constraint, out e2)) {              model.Adjacent(constraint.IsHard, data.Rooms, e1, e2);            }            break;            case ConstraintType.AwayFrom:            if (TryGetSubjectEntity(data, constraint, out e1) && TryGetTargetEntity(data, constraint, out e2)) {              model.AwayFrom(constraint.IsHard, data.Rooms, e1, e2);            }            break;            default:            throw new InvalidOperationException("Invalid constraint.");          }        }      }        private void Report(OfficeSpaceData data, Dictionary<ConstraintType, double> penalties, Solution solution) {        ReportSection("WEIGHTS:");        foreach (var pair in penalties) {          Console.WriteLine("{0}\t{1}", pair.Key, pair.Value);        }          if (solution.Quality.StartsWith("infeasible", StringComparison.OrdinalIgnoreCase)) {          Console.WriteLine("*** MODEL IS INFEASIBLE ***");          return;        }          if (printAssignment) {          ReportSection("ASSIGNMENT:");          Console.WriteLine("ENTITY ROOM");          foreach (var assignment in solution.EntityToRoom.OrderBy(p => p.Key)) {            Console.WriteLine("{0} {1}", assignment.Key, assignment.Value);          }        }          ReportSection("USAGE TERMS:");        foreach (var entry in solution.Usage) {          string r = entry.Item1;          Console.WriteLine("{0} = {1:f4}\tRoom = {2}\tEntity = {3} (count = {4})", r, entry.Item2,             data.RoomsByID[r].Size,            solution.EntityToRoom.Where(t => t.Value == r).Sum(t => data.EntitiesByName[t.Key].Space),            solution.EntityCount(r)            );        }        Console.WriteLine("TOTAL USAGE = " + solution.Usage.Sum(u => u.Item2));                ReportSection("SUMMARY:");        Console.WriteLine("OBJECTIVE = " + solution.Objective);        Console.WriteLine("COST      = " + Evaluate(data, penalties, solution.EntityToRoom));      }        private static void ReportSection(string label) {        Console.WriteLine(Environment.NewLine + label);        Console.WriteLine(new string('-', label.Length));      }        private static double Evaluate(OfficeSpaceData data,         Dictionary<ConstraintType, double> penalties,        IDictionary<string, string> entityToRoom) {        SolutionEvaluator e = new SolutionEvaluator(data, entityToRoom, penalties);        double cost = 0;        string e1, e2, room;        foreach (var constraint in data.Constraints) {          switch (constraint.Type) {          case ConstraintType.NoSharing:            if (TryGetSubjectEntity(data, constraint, out e1)) {              cost += e.NoSharingCost(e1);            }            break;            case ConstraintType.AllocateTo:            if (TryGetSubjectEntity(data, constraint, out e1) && TryGetTargetRoom(data, constraint, out room)) {              cost += e.AllocateToCost(e1, room);            }            break;            case ConstraintType.GroupBy:            if (TryGetSubjectEntity(data, constraint, out e1) && TryGetTargetEntity(data, constraint, out e2)) {              cost += e.GroupByCost(e1, e2);            }            break;            case ConstraintType.SameRoom:            if (TryGetSubjectEntity(data, constraint, out e1) && TryGetTargetEntity(data, constraint, out e2)) {              cost += e.SameRoomCost(e1, e2);            }            break;            case ConstraintType.Adjacent:            if (TryGetSubjectEntity(data, constraint, out e1) && TryGetTargetEntity(data, constraint, out e2)) {              cost += e.AdjacentCost(e1, e2);            }            break;            case ConstraintType.AwayFrom:            if (TryGetSubjectEntity(data, constraint, out e1) && TryGetTargetEntity(data, constraint, out e2)) {              cost += e.AwayFromCost(e1, e2);            }            break;            default:            throw new InvalidOperationException("Invalid constraint.");          }        }        cost += data.Rooms.Sum(r => e.RoomUsageCost(r.ID));          return cost;      }        private static bool TryGetSubjectEntity(OfficeSpaceData data, Constraint constraint, out string entity) {        entity = constraint.Subject;        if (!data.EntitiesByName.ContainsKey(constraint.Subject)) {          Console.WriteLine(String.Format("*** Invalid subject entity [{0}] in row {1}", entity, constraint));          return false;        }        return true;      }        private static bool TryGetTargetEntity(OfficeSpaceData data, Constraint constraint, out string entity) {        entity = constraint.Target;        if (!data.EntitiesByName.ContainsKey(entity)) {          Console.WriteLine(String.Format("*** Invalid target entity [{0}] in row {1}", entity, constraint));          return false;        }        return true;      }        private static bool TryGetTargetRoom(OfficeSpaceData data, Constraint constraint, out string room) {        room = constraint.Target;        if (!data.RoomsByID.ContainsKey(room)) {          Console.WriteLine(String.Format("*** Invalid target room [{0}] in row {1}", room, constraint));          return false;        }        return true;      }    }  }

Here is the output of the program for my sample problem (you should be able to confirm this by downloading the Express edition of Solver Foundation). As you can see, the optimal cost is 62. If you look at the assignment, the “Form” team is allocated to the first floor and the “Function” team is allocated to the second floor. Also notice that some of the rooms on the first floor are unused! This is because the sample model has constraints that specify that teammates should be “nearby” each other, which means allocated to the same floor. Since this constraint is strictly enforced this effectively means that each team must be on a separate floor. If we did not have such a strict interpretation of the “nearby” constraints then perhaps we could find an allocation that had better space usage. In order to support soft constraints we will need to revisit the model class, which we will do in the next post.

c:\temp\OfficeSpace\bin\Release>OfficeSpace.exe \temp\data\nott_data\sample.xml  FILE: \temp\data\nott_data\sample.xml  PROCESSING 11 CONSTRAINTS  Optimize a model with 227 Rows, 326 Columns and 1853 NonZeros  Presolve removed 109 rows and 227 columns  Presolve time: 0.03s  Presolved: 118 Rows, 99 Columns, 845 Nonzeros  Objective GCD is 1  Found heuristic solution: objective 104.0000    Root relaxation: objective 2.401047e+01, 133 iterations, 0.00 seconds        Nodes    |    Current Node    |     Objective Bounds      |     Work   Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time         0     0    24.0105    0   41   104.0000    24.0105  76.9%     -    0s       0     0    41.7082    0   33   104.0000    41.7082  59.9%     -    0s       0     0    41.8302    0   40   104.0000    41.8302  59.8%     -    0s       0     0    41.8302    0   37   104.0000    41.8302  59.8%     -    0s  H    0     0                         62.0000    41.8302  32.5%     -    0s       0     0    47.5859    0   34    62.0000    47.5859  23.2%     -    0s       0     0    47.5859    0   37    62.0000    47.5859  23.2%     -    0s       0     0    47.5859    0   36    62.0000    47.5859  23.2%     -    0s       0     0    47.5859    0   36    62.0000    47.5859  23.2%     -    0s       0     2    47.5859    0   34    62.0000    47.5859  23.2%     -    0s    Cutting planes:    Implied bound: 3    MIR: 7    Explored 16 nodes (595 simplex iterations) in 0.11 seconds  Thread count was 2 (of 2 available processors)    Optimal solution found (tolerance 1.00e-04)  Best objective 6.2000000000e+01, best bound 6.2000000000e+01, gap 0.0%    WEIGHTS:  --------  Adjacent        10  AllocateTo      20  AwayFrom        10  NoSharing       50  SameRoom        10  GroupBy 11.18  Wastage 1  Overuse 2    ASSIGNMENT:  -----------  ENTITY ROOM  Alexander 102  Banksy 101  Christo 203  David 202  Einstein 201  Fabio 105  Galileo 202  Holmes 202  Icarus 103  Jian 202    USAGE TERMS:  ------------  101 = 4.0000    Room = 8        Entity = 10 (count = 1)  102 = 18.0000   Room = 6        Entity = 15 (count = 1)  103 = 2.0000    Room = 6        Entity = 4 (count = 1)  104 = 6.0000    Room = 6        Entity = 0 (count = 0)  106 = 6.0000    Room = 6        Entity = 0 (count = 0)  201 = 18.0000   Room = 6        Entity = 15 (count = 1)  202 = 6.0000    Room = 22       Entity = 25 (count = 4)  203 = 2.0000    Room = 14       Entity = 15 (count = 1)  TOTAL USAGE = 62    SUMMARY:  --------  OBJECTIVE = 62  COST      = 62

If you are exploring the Windows Azure AppFabric Service Bus capabilities and trying the bits out in your projects, the following article may shed some light on where to be on a lookout when it comes to using the netTcpRelayBinding in Streaming mode.

Scenario

I’m currently leading the development efforts on a customer project where we are streaming large amounts of structured (XML) data from on-premise BizTalk Server 2010 environment all the way to a cloud-based inventory database hosted on SQL Azure. The message flow can be simplified to an extent where it can be described as follows:

1. Inventory files are being received from many EDI partners and transformed into a canonical inventory schema representation using BizTalk Server’s support for EDI interoperability and data mapping/transformation;
2. The canonical inventory schema instances are being picked up by a designated WCF-Custom Send Port configured with netTcpRelayBinding that talks to the Azure Service Bus;
3. The inventory data is relayed in streaming mode through the Service Bus to a WCF Service endpoint hosted in a worker role on the Windows Azure;
4. The WCF Service receives the data stream and relays it further to a SQL Azure database-based queue so that the data becomes available for processing.

Below is the depicted version of the message flow that we have implemented at the initial stage of the project:

The Windows Azure AppFabric Service Bus make the above scenario shine as it makes it easy to connect the existing on-premise BizTalk infrastructure with cloud-based service endpoints. While it’s truly an eye-opener, there are several observations that we have made as it relates to data streaming over TCP sockets.

Observations

As referenced above, the cloud-hosted WCF service exposes a streaming-aware operation that takes the inbound data stream and makes sure that it safely lands in a SQL Azure database. Specifically, we are reading the data from inbound stream into a memory buffer in chunks and then flush the buffer’s content into a  varchar(max) field using the Write() mutator operation supported by the UPDATE command.

The code snippet implementing the above technique is shown below:

#region IPersistenceServiceContract implementation  public Guid PersistDataStream(Stream data)  {      // Some unrelated content was omitted here and the code below was intentionally simplified for sake of example.


    // For best performance, we recommend that data be inserted or updated in chunk sizes that are 
    // multiples of 8040 bytes.      int bufferSize = 8040 * 10;        using (ReliableSqlConnection dbConnection = new ReliableSqlConnection(dbConnectionString))      using (SqlStream sqlStream = new SqlStream(dbConnection, readDataCommand, writeDataCommand, getDataSizeCommand))      {          BinaryReader dataReader = new BinaryReader(data);            byte[] buffer = new byte[bufferSize];          int bytesRead = 0;            do          {              bytesRead = dataReader.Read(buffer, 0, bufferSize);                if (bytesRead > 0)              {                  TraceManager.CustomComponent.TraceInfo("About to write {0} bytes into SQL Stream", bytesRead);                    sqlStream.Write(buffer, 0, bytesRead);              }          }          while (bytesRead > 0);      }        return Guid.NewGuid();  }  #endregion

Despite the fact that both the client and server WCF bindings were configured correctly and identically as and where appropriate, including such important configuration parameters as reader quotas, max buffer size, etc, we have noticed that the specified buffer size was not appreciated by the underlying WCF stream. This basically means that the chunk size returned from the Read method was never ever near the anticipated buffer size of 80400 bytes. The following trace log fragment supports the above observations (note the instrumentation event in the above code that we emit before writing data into a SQL database):

There is an explanation for the behavior in question.

First of all, some fluctuation is the read chunk size bubbled up by the OSI transport layer is expected on any TCP socket connection. With TCP streaming, the data is being made available immediately as it streams off the wire. The TCP sockets generally don’t attempt to fill the buffer in full, they do their best to supply as much data as they can as timely as they can.

Secondly, when we set the buffer size to 80400 bytes, we unintentionally attempted to ask the TCP stack to buffer up to 53 times of its Maximum Transmission Unit (MTU) value as well to potentially exceed the maximum TCP receive window size. This is an unrealistic ask.

So, why do these small incremental (sometimes appearing to be random) chunks project potential concerns to a developer? Well, in our example, we are writing data into a SQL Azure database and we want this operation to be as optimal as possible. Writing 2, 6, 255 or even 4089 bytes per each call doesn’t allow us to achieve the appropriate degree of efficiency. Luckily, a solution for this challenge comes across extremely well in the following simple approach.

Solution

Simply put, we need to make sure that the data will be continuously read from the inbound stream into a buffer until the buffer is full. This means that we will not stop after the first invocation of the Read method - we will be repetitively asking the stream to provide us with the data until we satisfied that we have received the sufficient amount. The easiest way of implementing this would be through an extension method in C#:

public static class BinaryReaderExtensions  {      public static int ReadBuffered(this BinaryReader reader, byte[] buffer, int index, int count)      {          int offset = 0;            do          {              int bytesRead = reader.Read(buffer, index + offset, count);                if (bytesRead == 0)              {                  break;              }                offset += bytesRead;              count -= bytesRead;          }          while (count > 0);            return offset;      }  }

Now we can flip the method name from Read to ReadBuffered in the consumer code leaving the rest unchanged:

        do          {
            // Note the name changed from Read to ReadBuffered as we are now using the extension method.              bytesRead = dataReader.ReadBuffered(buffer, 0, bufferSize);                if (bytesRead > 0)              {                  TraceManager.CustomComponent.TraceInfo("About to write {0} bytes into SQL Stream", bytesRead);                    sqlStream.Write(buffer, 0, bytesRead);              }          }          while (bytesRead > 0);

The end result is that we can now guarantee that each time we invoke a SQL command to write data into a varchar(max) field, we deal with fully filled-in buffers and data chunks the size of which we can reliably control:

As an extra benefit, we reduced the number of database transactions since we are now able to stream larger chunks of data as opposed to invoking the SQL command for a number of smaller chunks as it was happening before.

Conclusion

Streaming is a powerful and high-performance technique for large data transmission. Putting on the large Azure sun glasses, we can confidently say that the end-to-end streaming between on-premise applications and the Cloud unlocks extremely interesting scenarios that makes impossible possible.

In this article, we shared some observations from our recent Azure customer engagement and provided recommendations as to how to avoid a specific “gotcha” with WCF streaming over netTcpRelayBinding in the Windows Azure AppFabric Service Bus. When implemented, the recommendations help ensure that developers have control over the size of the data that comes out of the underlying WCF stream and reduce inefficiency associated with “smaller than expected” data chunks.

Additional Resources/References

For more information on the related topic, please visit the following resources:

• “Large Message Transfer with WCF-Adapters Part 1” article by Paolo Salvatori;
• “WCF Streams” article in the MSDN Library;
• “Large Data and Streaming” article in the MSDN Library;
• “How to Enable Streaming” article in the MSDN Library;

Hello world. I was out of the office late last week so there was no mail sack; Jonathan pretended like he was going to do one but he lied. He’ll try to claim that things got “busy” and there were “customers” who wanted “their issues fixed” or some other nonsense, but we all know it was due to him daydreaming about bubble baths.

Too weird?

Anyway, what with the hiring we’re doing now, a month ago I promised you some further reading around how you can amp up your Active Directory skills. Rather than burying it in another mail sack, I figured I’d lay it all out here in one spot. If you feel like you need to fill in the cracks on your directory service knowledge, here’s what we force feed our new hires:

Core Technology Reading

If you read nothing else, read these core pieces. While they are Win2003/XP specific, that’s still at least 75% of the business install base and highly relevant. For the most part things don’t change that much architecturally between versions either (ignoring GP and User Profiles). They give you the fundamentals to build on later.

Active Directory Collection
Active Directory Replication Model
Active Directory Replication Topology
Authentication
Authorization
DNS Technical Reference
Group Policy
Interactive Logon
Kerberos Authentication Technical Reference
Public Key Infrastructure (PKI)
TCP/IP Technical Reference
User Profiles

Post Graduate Technology Reading

Then we get to the more advanced subjects, the specific features added in later models, and the things that will take you into rarefied air. Much of this is Windows Server 2008 and later too, so if you haven’t started rolling out our later OS this will get you ready. If you can get through these, you’re ready to run AD in the environments with 100,000+ computers. And as I always tell people, if you know how something works, you can troubleshoot any kind of problem – even if the issue has never seen seen before.

Active Directory Domain Services in the Perimeter Network
Active Directory and Active Directory Domain Services Port Requirements
Active Directory Schema
ADMT Guide: Migrating and Restructuring Active Directory Domains
AppLocker
AD DS Design Guide
CA Certificates
Certificates
Certificate Services
Core Group Policy Technical Reference
Designing a Group Policy Infrastructure
DFSR
DFS Replication: Frequently Asked Questions (FAQ)
Distributed File System (DFS)
DNS Support for Active Directory
Domain and Forest Trusts Technical Reference
File Replication Service FRS
Global Catalog Technical Reference 
Group Policy Components
Group Policy Management Console
Group Policy Object Editor
Logon and Authentication Technologies
Managed Service Accounts
Managing Roaming User Data Deployment Guide
Operations Masters Technical Reference
Read-Only Domain Controller Planning and Deployment Guide
Running Domain Controllers in Hyper-V
Security Auditing
Security Compliance Manager
Security Identifiers Technical Reference
Security Descriptors and Access Control Lists Technical Reference
Security Principals Technical Reference
Staging Group Policy Deployments
SYSVOL Replication Migration Guide: FRS to DFS Replication
User Account Control Technical Reference
What's New in Active Directory Domain Services in Win2008
What's New in Active Directory Domain Services in Win2008 R2
Windows Smart Card Technical Reference
Windows Time Service Technical Reference
WINS Technical Reference

Lab Materials

You can use these free trial editions below in order to do live repros of all this, and repros are highly suggested. Especially with the use of Netmon 3.4 to see how things look on the wire and learn how we troubleshoot here – with network captures. Running these in Hyper-V, in Virtualbox, etc. will also make the materials more understandable.

http://www.microsoft.com/windowsserver2008/en/us/trial-software.aspx
http://technet.microsoft.com/en-us/evalcenter/cc442495.aspx

As an alternative, for a few hundred bucks you can get the amazingly packed TechNet or MSDN subscriptions that provide you with copies of so much MS software it’s ridiculous; way better than using trialware. Check those out here:

http://technet.microsoft.com/en-us/subscriptions/buy.aspx
http://msdn.microsoft.com/en-us/subscriptions/buy.aspx

Thanks to the Blue Devil who reminded me to do this. :-)

Ned “nutty professor” Pyle

I recently needed to import an Access database into a C# program for a sample that I will be blogging about soon. My objective was to convert the data into a more convenient form for use with my “real” application. Nothing here will be very cutting edge! A quick and dirty way to get the job done seemed to be to read the data into a DataSet and export it to XML. Access MDB files can be read using the Jet OLDB provider with OleDbConnection. Once the connection is established, the GetOleDbSchemaTable method can be used to get the table names. Then each table can be read using a select. Writing the data out to XML is easy using the built-in DataSet.WriteToXml() method. I also write out the schema file so that the columns will have the correct types when I read the data back in.

One last hitch: in .Net 4/VS2010 the OLEDB component works only with a 32-bit build. So change the Platform target in the Project properties as follows:

Here's the code. I’ve made even less of an effort than usual to make the code “production quality”, but note that a couple of the classes I use are IDisposable so I’m taking care to wrap them in a “using” block.

using System;  using System.Collections.Generic;  using System.Data;  using System.Data.OleDb;  using System.Linq;  using System.Text;    // None of this is foolproof...caveat emptor.  namespace AccessToDataSet {    class Program {      static void Main(string[] args) {        if (args.Length == 0 || !args[0].EndsWith(".mdb", StringComparison.InvariantCultureIgnoreCase)) {          Console.WriteLine("Please specify the path to an MDB file.");          return;        }          DataSet dataSet = new DataSet();        using (var conn = new OleDbConnection(@"Provider=Microsoft.JET.OLEDB.4.0;" + @"data source=" + args[0])) {          conn.Open();          // Retrieve the schema          DataTable schemaTable = conn.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, new object[] { null, null, null, "TABLE" });          // Fill the DataTables.          foreach (DataRow dataTableRow in schemaTable.Rows) {            string tableName = dataTableRow["Table_Name"].ToString();            // I seem to get an extra table starting with ~. I can't seem to screen it out based on information in schemaTable,            // hence this hacky check.            if (!tableName.StartsWith("~", StringComparison.InvariantCultureIgnoreCase)) {              FillTable(dataSet, conn, tableName);            }          }        }          string name = args[0].ToLowerInvariant();        dataSet.WriteXmlSchema(name.Replace(".mdb", ".schema.xml"));        dataSet.WriteXml(name.Replace(".mdb", ".xml"));      }        private static void FillTable(DataSet dataSet, OleDbConnection conn, string tableName) {        DataTable dataTable = dataSet.Tables.Add(tableName);        using (OleDbCommand readRows = new OleDbCommand("SELECT * from " + tableName, conn)) {          OleDbDataAdapter adapter = new OleDbDataAdapter(readRows);          adapter.Fill(dataTable);        }      }    }  }

In the world of relational databases circular references are schema structures where foreign keys relating the tables create a loop. Circular references cause special types of issues when trying to synchronize two relational database where the foreign keys are enforced. Because of this issue, database schemas that contain circular references are restricted in the tools that can be used when synchronizing and replicating the database. This article will explain circular references and demonstrate a Transact-SQL script for determining if your database has a circular reference.

What is a Circular Reference?

Foreign keys create database-enforced integrity constraints. These constraints ensure that a row of data exists in one table before another table can reference it. They also prevent a dependent row from being deleted that another row references. In Figure 1 we see a simple foreign key between Address table and StateProvince table in the Adventure Works database.

Figure 1

A circular reference is one or more tables where the foreign keys create a loop. Figure 2 is an example.

Figure 2

In this case the City table contains a reference to the author; it is the author that wrote the description for the city. The Author table has a reference to the city, because each author lives in a city. So which came first, the city or the author? In all cases with circular references one of the foreign key columns must be accept a null value. This allows the data to be inserted in 3 passes:

1. An insert into the table referenced by the nullable foreign key with the key set to null.
2. An insert into the table with the non-null foreign key.
3. An update to modify the nullable foreign key to reference the row inserted in step 2.

A circular reference is not limited to two tables, it might involve many tables, all bound together in one big circle.

Self-Referencing Tables

A special case circular reference is the self-referencing table. This is a table that has a foreign key column that references its own primary key. An example is a human resource schema that tracks employees and their bosses. In the employee table, there is a foreign key column called boss that references the primary key column in the employee table. Self-referencing tables always have a foreign key column which is nullable and at least one null exists. In the example above it would be the CEO, since he doesn’t have a boss his boss column is null.

Synchronizing Schemas with Circular References

Tables that are not involved in a circular reference are easy to synchronize, you make a complete table update the table without dependencies on it, then update the tables with foreign key dependences. In Figure 1 you would update the StateProvince table, then the Address table. This explanation is simplified, for example the deletes are done in the reverse order. If the tables have no circular references you can synchronize them table by table if you know their dependency order.

Synchronizing tables with circular references is much harder, because you have to update the tables row by row, jumping back and forth between the tables, inserting the nullable foreign key with nulls first, then updating them later. Again this is a simplified explanation; the point is that you can’t update the tables in a serial order if there are circular references.

There are really only a couple ways to synchronize database that contains tables with circular references:

• Perform a transaction based replication, much like SQL Server replication, which updates, inserts, and deletes the data in the same serial order as the data was changed in the source database
• Set the database into read-only mode, bulk copy the rows over to the destination database with the same primary keys, without check constraints on. Once you have moved all the tables, the source database can be taken out of read-only mode. I blog about doing this with bcp utility here.
• Deduce the possible orders of inserts, updates, and deletes row by row based on the dependencies and recreate those on the destination database. This is comparable to backwards engineering the transactions it took to update, insert and delete the data.

Detecting Circular References

The Transact-SQL script below uses a recursive cursor to detect if there are any circular references in your database schema. It can be run on your SQL Server database before you try to synchronize it with SQL Azure, or you can run it on your SQL Azure database. You can run it in the Query Window of SQL Server Management Studio; the output will be displayed as in the Message section.

If you have circular references the output will look like this:

dbo.City -> dbo.Author -> dbo.City  dbo.Division -> dbo.Author -> dbo.City -> dbo.County -> dbo.Region -> dbo.Image -> dbo.Division  dbo.State -> dbo.Image -> dbo.Area -> dbo.Author -> dbo.City -> dbo.County -> dbo.Region -> dbo.State  dbo.County -> dbo.Region -> dbo.Author -> dbo.City -> dbo.County  dbo.Image -> dbo.Area -> dbo.Author -> dbo.City -> dbo.County -> dbo.Region -> dbo.Image  dbo.Location -> dbo.Author -> dbo.City -> dbo.County -> dbo.Region -> dbo.Image -> dbo.Location  dbo.LGroup -> dbo.LGroup  dbo.Region -> dbo.Author -> dbo.City -> dbo.County -> dbo.Region  dbo.Author -> dbo.City -> dbo.Author  dbo.Area -> dbo.Author -> dbo.City -> dbo.County -> dbo.Region -> dbo.Image -> dbo.Area

Each line is a circular reference, with a link list of tables that create the circle.

SET NOCOUNT ON    -- WWB: Create a Temp Table Of All RelationShip To Improve Overall Performance  CREATE TABLE #TableRelationships (FK_Schema nvarchar(max), FK_Table nvarchar(max),      PK_Schema nvarchar(max), PK_Table nvarchar(max))    -- WWB: Create a List Of All Tables To Check  CREATE TABLE #TableList ([Schema] nvarchar(max), [Table] nvarchar(max))    -- WWB: Fill the Table List  INSERT INTO #TableList ([Table], [Schema])  SELECT TABLE_NAME, TABLE_SCHEMA  FROM INFORMATION_SCHEMA.TABLES   WHERE Table_Type = 'BASE TABLE'    -- WWB: Fill the RelationShip Temp Table  INSERT INTO #TableRelationships(FK_Schema, FK_Table, PK_Schema, PK_Table)  SELECT      FK.TABLE_SCHEMA,      FK.TABLE_NAME,      PK.TABLE_SCHEMA,      PK.TABLE_NAME  FROM INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS C        INNER JOIN INFORMATION_SCHEMA.TABLE_CONSTRAINTS FK ON           C.CONSTRAINT_NAME = FK.CONSTRAINT_NAME        INNER JOIN INFORMATION_SCHEMA.TABLE_CONSTRAINTS PK ON           C.UNIQUE_CONSTRAINT_NAME = PK.CONSTRAINT_NAME        INNER JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE CU ON           C.CONSTRAINT_NAME = CU.CONSTRAINT_NAME        INNER JOIN (              SELECT i1.TABLE_NAME, i2.COLUMN_NAME              FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS i1              INNER JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE i2 ON               i1.CONSTRAINT_NAME = i2.CONSTRAINT_NAME              WHERE i1.CONSTRAINT_TYPE = 'PRIMARY KEY'  ) PT ON PT.TABLE_NAME = PK.TABLE_NAME    CREATE TABLE #Stack([Schema] nvarchar(max), [Table] nvarchar(max))    GO    -- WWB: Drop SqlAzureRecursiveFind  IF  EXISTS (SELECT * FROM sys.objects WHERE object_id =       OBJECT_ID(N'[dbo].[SqlAzureRecursiveFind]') AND type in (N'P', N'PC'))  DROP PROCEDURE [dbo].[SqlAzureRecursiveFind]    GO    -- WWB: Create a Stored Procedure that Recursivally Calls Itself  CREATE PROC SqlAzureRecursiveFind      @BaseSchmea nvarchar(max),      @BaseTable nvarchar(max),      @Schmea nvarchar(max),      @Table nvarchar(max),      @Fail nvarchar(max) OUTPUT  AS        SET NOCOUNT ON                      -- WWB: Keep Track Of the Schmea and Tables We Have Checked      -- Prevents Looping                INSERT INTO #Stack([Schema],[Table]) VALUES (@Schmea, @Table)            DECLARE @RelatedSchema nvarchar(max)      DECLARE @RelatedTable nvarchar(max)            -- WWB: Select all tables that the input table is dependent on      DECLARE table_cursor CURSOR LOCAL  FOR            SELECT PK_Schema, PK_Table            FROM #TableRelationships            WHERE FK_Schema = @Schmea AND FK_Table = @Table        OPEN table_cursor;        -- Perform the first fetch.      FETCH NEXT FROM table_cursor INTO @RelatedSchema, @RelatedTable;        -- Check @@FETCH_STATUS to see if there are any more rows to fetch.      WHILE @@FETCH_STATUS = 0      BEGIN                -- WWB: If We have Recursed To Where We Start This          -- Is a Circular Reference          -- Begin failing out of the recursions          IF (@BaseSchmea = @RelatedSchema AND                   @BaseTable = @RelatedTable)              BEGIN                  SET @Fail = @RelatedSchema + '.' + @RelatedTable                  RETURN              END          ELSE                      BEGIN                        DECLARE @Count int                        -- WWB: Check to make sure that the dependencies are not in the stack              -- If they are we don't need to go down this branch              SELECT    @Count = COUNT(1)              FROM    #Stack                  WHERE    #Stack.[Schema] = @RelatedSchema AND                   #Stack.[Table] = @RelatedTable                        IF (@Count=0)               BEGIN                  -- WWB: Recurse                  EXECUTE SqlAzureRecursiveFind @BaseSchmea,                       @BaseTable,                       @RelatedSchema, @RelatedTable, @Fail OUTPUT                  IF (LEN(@Fail) > 0)                  BEGIN                      -- WWB: If the Call Fails, Build the Output Up                      SET @Fail = @RelatedSchema + '.' + @RelatedTable                           + ' -> ' + @Fail                      RETURN                  END              END         END                      -- This is executed as long as the previous fetch succeeds.      FETCH NEXT FROM table_cursor INTO @RelatedSchema, @RelatedTable;      END        CLOSE table_cursor;      DEALLOCATE table_cursor;        GO        SET NOCOUNT ON    DECLARE @Schema nvarchar(max)  DECLARE @Table nvarchar(max)  DECLARE @Fail nvarchar(max)    -- WWB: Loop Through All the Tables In the Database Checking Each One  DECLARE list_cursor CURSOR FOR        SELECT [Schema], [Table]        FROM #TableList    OPEN list_cursor;    -- Perform the first fetch.  FETCH NEXT FROM list_cursor INTO @Schema, @Table;    -- Check @@FETCH_STATUS to see if there are any more rows to fetch.  WHILE @@FETCH_STATUS = 0  BEGIN        -- WWB: Clear the Stack (Don't you love Global Variables)      DELETE #Stack            -- WWB: Intialize the Input      SET @Fail = ''        -- WWB: Check the Table      EXECUTE SqlAzureRecursiveFind @Schema,           @Table, @Schema,           @Table, @Fail OUTPUT      IF (LEN(@Fail) > 0)      BEGIN          -- WWB: Failed, Output          SET @Fail = @Schema + '.' + @Table + ' -> ' + @Fail          PRINT @Fail      END                   -- This is executed as long as the previous fetch succeeds.      FETCH NEXT FROM list_cursor INTO @Schema, @Table;  END    -- WWB: Clean Up  CLOSE list_cursor;  DEALLOCATE list_cursor;                            DROP TABLE #TableRelationships  DROP TABLE #Stack  DROP TABLE #TableList  DROP PROC SqlAzureRecursiveFind

Summary

Do you have questions, concerns, comments? Post them below and we will try to address them.

SQL Azure currently supports 1 GB and 10 GB databases, and on June 28^th, 2010 there will be 50 GB support. If you want to store larger amounts of data in SQL Azure you can divide your tables across multiple SQL Azure databases. This article will discuss how to use a data access layer to join two tables on different SQL Azure databases using LINQ. This technique horizontal partitions your data in SQL Azure.

In our version of horizontal partitioning, every table exists in all the databases in the partition set. We are using a hash base partitioning schema in this example – hashing on the primary key of the row. The middle layer determines which database to write each row based on the primary key of the data being written. This allows us to evenly divide the data across all the databases, regardless of individual table growth. The data access knows how to find the data based on the primary key, and combines the results to return one result set to the caller.

This is considered hash based partitioning. There is another style of horizontal portioning that is range based. If you are using integers as primary keys you can implement your middle layer to fill databases in consecutive order, as the primary key grows. You can read more about the different types of partitioning here.

Performance Gain

There is also a performance gain to be obtained from partitioning your database. Since SQL Azure spreads your databases across different physical machines, you can get more CPU and RAM resources by partitioning your workload. For example, if you partition your database across 10 - 1 GB SQL Azure databases you get 10X the CPU and memory resources. There is a case study (found here) by TicketDirect, who partitioning their workload across hundreds of SQL Azure databases during peak load.

Considerations

When horizontal partitioning your database you lose some of the features of having all the data in a single database. Some considerations when using this technique include:

• Foreign keys across databases are not supported. In other words, a primary key in a lookup table in one database cannot be referenced by a foreign key in a table on another database. This is a similar restriction to SQL Server’s cross database support for foreign keys.
• You cannot have transactions that span two databases, even if you are using Microsoft Distributed Transaction Manager on the client side. This means that you cannot rollback an insert on one database, if an insert on another database fails. This restriction can be mitigated through client side coding – you need to catch exceptions and execute “undo” scripts against the successfully completed statements.
• All the primary keys need to be uniqueidentifier. This allows us to guarantee the uniqueness of the primary key in the middle layer.
• The example code shown below doesn’t allow you to dynamically change the number of databases that are in the partition set. The number of databases is hard coded in the SqlAzureHelper class in the ConnectionStringNames property.
• Importing data from SQL Server to a horizontally partitioned database requires that you move each row one at a time emulating the hashing of the primary keys like the code below.

The Code

The code will show you how to make multiple simultaneous requests to SQL Azure and combine the results to take advantage of those resources. Before you read this post you should familiarize yourself with our previous article about using Uniqueidentifier and Clustered Indexes and Connections and SQL Azure. In order to accomplish horizontal partitioning, we are using the same SQLAzureHelper class as we used in the vertical partitioning blog post.

The code has these goals:

• Use forward only cursors to maximize performance.
• Combine multiple responses into a complete response using Linq.
• Only access one database for primary key requests.
• Evenly divide row data across all the databases.

Accounts Table

The example table I am using is named Accounts and has a primary key of a uniqueidentifier, with a clustered index built on the Date column. I created it with this script on two databases:

CREATE TABLE [dbo].[Accounts](      [Id] [uniqueidentifier] NOT NULL,      [Name] [nvarchar](max) NULL,      [Date] [datetime] NULL,   CONSTRAINT [PK_Accounts] PRIMARY KEY NONCLUSTERED   (      [Id] ASC  )  )      ALTER TABLE [dbo].[Accounts] ADD  CONSTRAINT [DF__Accounts__Date__7C8480AE]  DEFAULT (getdate()) FOR [Date]      CREATE CLUSTERED INDEX [idxDate] ON [dbo].[Accounts]   (      [Date] ASC  )

Partitioning by Primary Key

Because the table we are partitioning is using uniqueidentifiers as its primary key, and these are generated by Guid.NewGuid(), we have a fairly random primary key. The primary key is hashed to figure out which database contains the row for that key. The code looks like this:

/// <summary>  /// Names of the Databases In Horizontal Partition  /// </summary>  public static String[] ConnectionStringNames =   { "Database001", "Database002" };    /// <summary>  /// Connections Strings In the Horizontal Partition  /// </summary>  /// <returns></returns>  public static IEnumerable<String> ConnectionStrings()  {      foreach (String connectionStringName in ConnectionStringNames)          yield return ConfigurationManager.  ConnectionStrings[connectionStringName].ConnectionString;  }    /// <summary>  /// Return the Index to the Database For the Primary Key  /// </summary>  /// <param name="primaryKey"></param>  /// <returns></returns>  private static int DatabaseIndex(Guid primaryKey)  {      return (BitConverter.ToInt32(primaryKey.ToByteArray(), 0));  }    /// <summary>  /// Returns the Connectiong String Name for the Primary Key  /// </summary>  /// <param name="primaryKey"></param>  /// <returns></returns>  private static String ConnectionStringName(Guid primaryKey)  {      return (ConnectionStringNames[DatabaseIndex(primaryKey) %   ConnectionStringNames.Length]);  }    /// <summary>  /// Returns the Connection String For the Primary Key  /// </summary>  /// <param name="primaryKey"></param>  /// <returns></returns>  public static String ConnectionString(Guid primaryKey)  {      return (ConfigurationManager.ConnectionStrings[ConnectionStringName(primaryKey)]  .ConnectionString);  } 

Notice that the ConnectionString() method returns the connection string to use when referencing the primary key. Configure the connection strings in the .config file for your application or web site. The array of databases referenced by the .config file is contained in the ConnectionStrings property.

Fetching A Single Row

I am building on top of the SQLAzureHelper class defined in the blog post. The idea behind the class is to have a multipurpose access layer to connect to SQL Azure. Within the SQLAzureHelper class, the code to find an account name in the example databases using the primary key looks like this:

static String AccountName(Guid id)  {      var accountDataReader = SQLAzureHelper.ExecuteReader(          SQLAzureHelper.ConnectionString(id),          sqlConnection =>          {              String sql = @"SELECT [Name] FROM [Accounts] WHERE Id = @Id";              SqlCommand sqlCommand = new SqlCommand(sql, sqlConnection);              sqlCommand.Parameters.AddWithValue("@Id", id);              return (sqlCommand.ExecuteReader());          });        return ((from row in accountDataReader select (string)row["Name"]).          FirstOrDefault());  }

Notice that we use the primary key to calculate the connection string and the as well as a parameter to the SqlCommand.

Inserting a Single Row

When inserting a single row, we need to know the primary key before connecting to the database. In order to accomplish this we call Guid.NewGuid() in the C# code, instead of NewID() on the SQL Azure. The code using the ExecutionContext class looks like this:

static Guid InsertAccount(String name)  {      Guid id = Guid.NewGuid();        SQLAzureHelper.ExecuteNonQuery(           SQLAzureHelper.ConnectionString(id),           sqlConnection =>           {               String sql = @"INSERT INTO [Accounts] ([Id], [Name]) VALUES (@Id, @Name)";               SqlCommand sqlCommand = new SqlCommand(sql, sqlConnection);               sqlCommand.Parameters.AddWithValue("@Name", name);               sqlCommand.Parameters.AddWithValue("@Id", id);               sqlCommand.ExecuteNonQuery();           });        return (id);  }

Summary

In part three, I will show how to fetch a result set that is merged from multiple responses and how to insert multiple rows into the partitioned tables, including some interesting multi-threaded aspects of calling many SQL Azure databases at the same time. Do you have questions, concerns, comments? Post them below and we will try to address them.

Last week the government released the massive Coins database. Find out what we know now
• Coins explorer

The Coins (Combined Online Information System) data release continues. Here are the key points so far:

1. More data

On Friday the Liberal-Conservative Coalition government released another three years of data - which means we now have every financial year from 2005/06 to 2009/10 - and you can use our Coins explorer to help navigate around the data

2. Overspends at the National Audit Office

The National Audit Office spent more than £60m on the refurbishment works to its Grade II listed art deco offices in Victoria, central London, which included marble flooring and leather sofas; and nearly £20m on temporary accommodation while the works were carried out, between 2008 - 2010. Some £2.33m was spent on furniture alone. The NAO is in charge of monitoring government spending.

3. The government's bad debts

Government was owed tens of millions of pounds in unpaid student loans and overpaid benefits last year.

4. COINS is too complicated

George Osborne announced that COINS would be redesigned next year to make it easier to use

5. The government spends a lot of money on consultants

£1.8bn spent on consultancy by government departments, up from £1.5bn in the previous year.

6. We now know a lot more about government spending

• Spending: £60bn public expenditure in March before the election - in August (the lowest month) it was £44.3bn
• European parliament: £6.8m spent on goods and services for members of the European parliament
• Swine flu: £100m tackling the flu pandemic
• Westminster: £25m net cost of the House of Commons
• Academies: £31.83m spent by the Department for Children, Schools and Families
• Lost legal fees: £111m Ministry of Justice provision for unbilled legal fees in October - up from £73m in April last year
• Counter-terrorism: £4m Office of security and counter terrorism payments to local authorities

7. The Guardian's specialists have been through the first release of data

You find out more on what our specialists found here.

8. The list of suppliers to one major government department

Courtesy of Where does my money go? we've seen that the only way to get detailed spending out of Whitehall is still to put in a Freedom of Information request. They put in a request to the Department for Environment Food and Rural Affairs to ask them how much they had spent on individual suppliers - which is missing from Coins. The results are fantastically detailed - we've put them on a spreadsheet which you can download here, or roll through them below.

They're asking for help to put in more requests so that we can fill in the Coins holes - get involved via a Google spreadsheet. Does this mean we're still going to need to perform copious FoI requests in future?

9. We know more about how the government categorises public spending

Lisa Evans at Where Does My Money Go? has also put together a list of programme object groups just for energy infrastructure, transport and tackling climate change. There's also an interesting freedom of information request now for more details about government department's bank accounts.

10. COINS is not comparable to other government finance publications

Lisa Evans writes about the the machinations of how the database came to be made public.

I'll tell you the story of the COINS database, which is a store of public spending and planning data, and you can judge for yourself how similar the story is to a gripping TV show.

The way COINS is described in documentation is that it's a store for government spending records. In the background to reports on public expenditure, like the PESA report, and many of the Office of National statistics reports on government spending, the guidance says the reports rely heavily on COINS data.

I asked for the COINS schema, but what I got was the dimensions of the OLAP database and no description of the fields meant, so that was the first mystery. Then I requests and got the COINS training notes, but with all the screen shots and lots of the descriptions redacted, so this was the second mystery.

When I met some people from the Treasury to talk about COINS some more and they told me about the thousands of spending codes in COINS, which I then requested, but with a number of them redacted, this was the third mystery.

Then, on Friday 4th June, we got the COINS data itself, well a sample from this year, and some COINS guidance(PDF) to go with it. The guidance says:

It is possible that you won't be able to recreate the numbers that Treasury or ONS have published. This is because:

• COINS has a single structure that is updated to reflect the latest classification of spend and organisation of government. The snapshots taken by the Treasury at defined points (e.g. to enable reports to be published) contain certain key fields, which then reference the latest structure. Any changes to the structure since the report was published may mean that it is not possible to recreate published figures;
• of the time difference between the publication of aggregated information and this release of data, with the potential that data have been updated between the differing points in time.
• Not all the data used to calculate these numbers are sourced from COINS.

So it's not possible to check the figures against the published reports, like PESA, to check we understand the data published. Another mystery.

So, effectively we can't compare Coins with other government data reports.

But, thanks to blogger Martin Budden - there may be a way of comparing. He has written two blog posts about COINS, one giving a brief overview of the COINS data format, and the second about how I used COINS to generate some of the PESA (Public Expenditure Statistical Analyses) reports. You can see more of what he's done with Coins at http://github.com/martinbudden/coins.

And, just for context, here's an interactive showing last year's overall spending by department.

Email us at coinsdata@guardian.co.uk if there's more you've found - or would like to know.

World government data

• Search the world's government data with our gateway

Can you do something with our data?

Flickr Please post your visualisations and mash-ups on our Flickr group or mail us at datastore@guardian.co.uk

• Get the A-Z of data
• More at the Datastore directory
• Follow us on Twitter

Are you floored by the recent Supreme Court decision unleashing corporate contributions to political campaigns?

Have you always suspected that voting in Congress is influenced by the contributions of interest groups, but never had the data to prove it?

Does a chance to work magic with that data to develop sophisticated tools to support high-profile research fill you with glee?

MAPLight.org, a cutting-edge non-partisan research organization located in Berkeley, California, is looking for the right stuff in a PHP/MySQL developer. We need someone who speaks in SQL and dreams in PHP. Someone who can look at gigabytes of data and quickly develop innovative strategies for getting the most out of our research and analysis tools.

Responsibilities:

*Work in a small, dynamic team to develop new web-based tools to support a talented research team and an audience of millions.
*Help to develop and optimize database schema and queries.
*Develop and improve processes to glean data from a wide range of data sources that is the backbone of our research.
*Enhance our Web 2.0 applications with top-notch PHP coding and client-side scripting.

Additional Qualifications:

*5+ years web development, with a strong focus on LAMP and open source tools.
*5+ years database experience in MySQL or Postgres.
*Very strong SQL skills.
*Expertise with PHP, DHTML/CSS/Javascript, XML.
*Knowledge of Drupal development.
*A knowledge of the American political system (or a desire to learn) is preferred, as is a passion for openness and transparency.

How to Apply:

This is a full-time position with salary and benefits commensurate with experience. MAPLight.org is located in downtown Berkeley, California (close to BART).

Interested candidates should submit a cover letter and resume to deanna [at] maplight [dot] org. Please include code samples and URLs to your online work.

About MAPLight.org:

MAPLight.org, a groundbreaking public database, illuminates the connection between campaign donations and legislative votes in unprecedented ways. Elected officials collect large sums of money to run their campaigns, and they often pay back campaign contributors with special access and favorable laws. This common practice is contrary to the public interest, yet legal. MAPLight.org makes money/vote connections transparent, to help citizens hold their legislators accountable.

MAPLight.org’s small, effective team has won numerous awards, including a 2008 Webby award nomination for best Politics website. Learn more at http://www.maplight.org.

Last week the government released the first part of the massive Coins database. Find out what we know now
• Coins explorer
• This article will update throughout today

It's only been three daya but are we getting a little blase about the tusnami of government data that's coming our way? On Friday, the government released part one of the massive Coins (Combined Online Information System) database to the public - you can download it here from our World Government Data search. They've since released data for 2008/09 too - we're working out how to get that onto our system now. You've been poring over the details using our Coins explorer all weekend.

Complicated as it is, it's shone a light on government spending. Here are some of the items we discovered on Friday:

• Bad debts: £596.4m owed to government departments
• Consultants: £1.8bn spent on consultancy by government departments, up from £1.5bn in the previous year
• Spending: £60bn public expenditure in March before the election - in August (the lowest month) it was £44.3bn
• European parliament: £6.8m spent on goods and services for members of the European parliament
• Swine flu: £100m tackling the flu pandemic
• Westminster: £25m net cost of the House of Commons
• Academies: £31.83m spent by the Department for Children, Schools and Families
• Lost legal fees: £111m Ministry of Justice provision for unbilled legal fees in October - up from £73m in April last year
• Counter-terrorism: £4m Office of security and counter terrorism payments to local authorities

You find out more on what our specialists found here.

Who do ministries buy equipment from?

One thing the release is done is highlight how much detailed information is actually missing from the database. Courtesy of Where does my money go? we've seen that the only way to get detailed spending out of Whitehall is still to put in a Freedom of Information request. They put in a requst to the Department for Environment Food and Rural Affairs to ask them how much they had spent on individual suppliers - which is missing from Coins. The results are fantastically detailed - we've put them on a spreadsheet which you can download here, or roll through them below.

They're asking for help to put in more requests so that we can fill in the Coins holes - get involved via a Google spreadsheet. Does this mean we're still going to need to perform copious FoI requests in future?

More today

• Lisa Evans at Where Does My Money Go? has also put together a list of programme object groups just for energy frastructure, transport and tackling climate change. Would be great to get more description of the codes and match them to spending.

• There's an interesting freedom of information request now for more details about government department's bank accounts.

• Lovely blogpost from Lisa too that's worth reading for the machinations of how the database came to be made public.

I'll tell you the story of the COINS database, which is a store of public spending and planning data, and you can judge for yourself how similar the story is to a gripping TV show.

The way COINS is described in documentation is that it's a store for government spending records. In the background to reports on public expenditure, like the PESA report, and many of the Office of National statistics reports on government spending, the guidance says the reports rely heavily on COINS data.

I asked for the COINS schema, but what I got was the dimensions of the OLAP database and no description of the fields meant, so that was the first mystery. Then I requests and got the COINS training notes, but with all the screen shots and lots of the descriptions redacted, so this was the second mystery.

When I met some people from the Treasury to talk about COINS some more and they told me about the thousands of spending codes in COINS, which I then requested, but with a number of them redacted, this was the third mystery.

Then, on Friday 4th June, we got the COINS data itself, well a sample from this year, and some COINS guidance(PDF) to go with it. The guidance says:

It is possible that you won't be able to recreate the numbers that Treasury or ONS have published. This is because:

• COINS has a single structure that is updated to reflect the latest classification of spend and organisation of government. The snapshots taken by the Treasury at defined points (e.g. to enable reports to be published) contain certain key fields, which then reference the latest structure. Any changes to the structure since the report was published may mean that it is not possible to recreate published figures;
• of the time difference between the publication of aggregated information and this release of data, with the potential that data have been updated between the differing points in time.
• Not all the data used to calculate these numbers are sourced from COINS.

So it's not possible to check the figures against the published reports, like PESA, to check we understand the data published. Another mystery.

So, effectively we can't compare Coins with other government data reports.

We're going to update this post throughout the day - not quite at live blog levels but enough to let you know things when we know them. Email us at coinsdata@guardian.co.uk if there's more you've found - or would like to know.

World government data

• Search the world's government data with our gateway

Can you do something with our data?

Flickr Please post your visualisations and mash-ups on our Flickr group or mail us at datastore@guardian.co.uk

• Get the A-Z of data
• More at the Datastore directory
• Follow us on Twitter

I love GUIDs -- the uniqueidentifier data type in SQL Server. I use them for everything, domain names, unique error message, and for primary keys in my database tables, etc… -- don’t get me started. About two years ago, I started using uniqueidentifier for primary keys because of their unique ability to merge.

Merging

When you create a GUID in SQL Server using the NewId() command you are guaranteed that it will be unique across the whole universe. Which means if you have two databases (with the same schema) completely disconnected adding rows to the same table, using a primary key of uniqueidentifier will ensure that they primary keys don’t conflict. In comparison, if the two databases had an identity integer column as the primary key for their tables, they would be very likely to insert the same primary key in both tables.

What does this have to do with merging? It is ridiculously easy to merge the tables of our disconnected databases with each other if they are using uniqueidentifier as their primary keys. On the other hand, it is much harder to resolve conflicts when merging between the identity integer primary keys and updating the foreign key references to those primary keys.

Clustering and uniqueidentifier

It isn’t a good idea to create a clustered index on a uniqueidentifier column and generate your GUIDs with NEWID(). The reason for this is that NEWID() generates GUIDs in non-sequential order and SQL Server orders a clustered index sequentially. It will work – SQL Server will let you build a clustered index around a uniqueidentifier column, however it will cause the SQL Server to do unnecessary work and cause performance slowdowns. The reason for this is that to insert data into the middle of a clustered index (out of sequential order) causes SQL Server to make room for the data by rearranging the cluster.

So if it isn’t a good idea then why do people do it? Well, in SQL Server, if I assign a column as the primary key in SQL Server Management Studio it automatically generates a clustered index for you regardless of the data type of that column. Therefore, if you want a table with a uniqueidentifier data type as a primary key you need to change that index to a non-clustered index.

Non-clustered indexes don’t reorder the data as rows are inserted to the table, so they don’t have the performance impact of a clustered index on inserts of non-sequential data.

Along comes SQL Azure

If you love uniqueidentifier data types for primary keys like I love them and you are creating non-clustered indexes on your primary keys then you need to pick a clustered index for the table also. The reason is that SQL Azure requires one (and only one) clustered index on all tables. However, just because SQL Azure requires a clustered index doesn’t mean it should be the primary key column.

Picking a Clustered Index

There are a couple of strategies for picking your clustered indexes; one of the easiest and best is to add another column of data type datetime and use that column for your clustered index. Here is what you need to do:

1. Add the column as data type datetime

2. I usually call it Date

3. Set the Default Value to GetDate().

4. Make it non-null.

5. Create your clustered index on it before you insert data into you.

Adding a default value of GetDate() to the column writes the date and time that the row was inserted into the column automatically. This insures that the data for the row is inserted at the end of the table data – there is no rearranging of the cluster. Adding the data to the end ensures the best performance for inserts.

Another good choice for a clustered index is a column that reflects the ordering of the table in the majority of the select statements. For example, if you have a table called categories and you have an integer column called ordervalue and you always call the table with an SELECT … ORDER BY [ordervalue] then making ordervalue the clustered index makes sense. Here is why: even though it hinders performance to insert a non-sequential ordervalue into the cluster you will get a performance benefit when you call your data, since the rows will be read sequential from the cluster, that it depending on your workload characteristics.

NEWSEQUENTIALID()

I would be remise if I didn’t mention the NEWSEQUENTIALID() function which not supported in SQL Azure. If the NEWID() function generates unique non-sequential uniqueidentifier than NEWSEQUENTIALID() function generates unique sequential uniqueidentifier. The only trick to NEWSEQUENTIALID() function is that the GUIDs are generated partial based on the network card of the computer.

This means that you can successfully have a uniqueidentifier as a primary key column and use that primary key column the required clustered index for SQL Server. As long as you use the NEWSEQUENTIALID() function column to fill that column.

If privacy is a concern, do not use this function. It is possible to guess the value of the next generated GUID and, therefore, access data associated with that GUID. For example if you are passing the primary key to the table in a query string on a web browser. See MSDN for more information.

However, NEWSEQUENTIALID() function isn’t support for SQL Azure. If you try to use it you will get this error:

Msg 40511, Level 15, State 1, Line 1

Built-in function 'NEWSEQUENTIALID' is not supported in this version of SQL Server.

Migration

If you are migrating an existing SQL Server database to SQL Azure you need to do these things to work successfully with uniqueidentifier data type as your primary key.

• Convert the generation of GUIDs from NEWSEQUENTIALID() to NEWID()
• Remove all clustered indexes from uniqueidentifier data type columns – this will not prevent you from migrating, however it will give you better performance.
• Pick a column (or add a column) to build your clustered index around (see the recommendations above).

Do you have questions, concerns, comments? Post them below and we will try to address them.

Job Title: Senior .NET Application Developer

Location: Woodridge, IL LOCAL CANDIDATES ONLY


Company Overview

Digital Aisle LLC is an industry-leading media company paving the way in next-generation digital shopper-marketing. The world’s largest Fortune retail companies rely on Digital Aisle because our proprietary solutions are revolutionizing the way to communicate with consumers at the critical point-of-purchase. The Digital Aisle platform has proven to deliver significant results, including higher consumer satisfaction and increased sales.

What makes us different? Digital Aisle knows the formula for success. We understand how to balance technological innovation with creative interactive content. The end result – consumers are more engaged and informed about their purchase decisions and our clients (retailers) achieve market share growth over their competitors.

How are we sure? With Digital Aisle’s proprietary software, it’s not about guessing, it’s about knowing. Our tracking system can be customized to measure consumer behavior and purchase decisions as they are happening. This enables retailers to respond to consumer trends more quickly than traditional tracking methods have historically allowed.




Job Description

Digital Aisle is seeking a local Chicago area candidate to fill a Senior .Net Application Developer position. This is a contract position based out of Woodridge, IL with minimal domestic travel. The desirable candidate must be highly motivated. The candidate must have experience designing and developing Desktop and Web applications using various Microsoft technologies with emphasis on rapid application development. The candidate must be able to work efficiently in a small team environment, be willing to learn new technologies, and will have extensive participation in the end-to-end software development process and regular client interaction.

Job Experience - Specific Skills

• At least 6-8 Years of Experience in using Microsoft .NET (versions 2.0 and 3.5) - C# and/or VB.NET to build Windows Forms based Desktop as well as ASP.NET based Web applications.
• Experience in the use of ADO.NET and ORM APIs such as NHibernate for Database Access
• Experience in developing Windows Native Services
• Experience in developing Web Services using the Windows Communication Framework
• Experience in use of Microsoft SQL Server (2005 and 2008), Oracle (9i or 10g), and MySQL – Entity Modelling, Transact SQL, Views, Stored procedures, Jobs, Triggers, etc.
• Experience in developing ActiveX Controls using either .NET or MFC/ATL
• XML Schema Design/XML Parsing/XSLT Development/XQuery/XPath

Job Experience – General Skills

• Proficient rapid application developer who can get going immediately
• Proficient in Designing and Implementing n-Tiered applications
• Proficient in Object Oriented Analysis and Design (OOAD) and in the use of various Design Patterns
• In-dept Understanding of Web Services and SOA concepts
• Experience with developing interoperability solutions for other technologies such as Java or Flash/Flex

Job Experience - Helpful Skills

• Experience with developing Flash/Flex Applications using ActionScript 2.0 and 3.0
• Exposure to Microsoft LINQ and The ADO.NET Entity Framework
• Experience in addressing security and transactions for Web Services
• MS SQL Server Admin and Performance Tuning Experience
• Experience in Monitoring and Tuning of Web Applications deployed on the Microsoft IIS Server versions 6.0 and 7.0
• Microsoft SQL Server Reporting & Analytics Services
• Experience with the use of the Microsoft ClickOnce Deployment Technology for distribution of application updates
• Exposure to Microsoft Azure Cloud Platform
• MCPD:Enterprise Application Developer Certification
• Exposure to Data Interchange Formats such as JSON in addition to XML and to understanding performance considerations arising from their use
• Exposure to use of various media types for video and still images in Desktop and Web Applications



Local Chicago Area Candidates not requiring sponsorship.

Digital Aisle LLC is an Equal Opportunity Employer

I'll be providing a more comprehensive description of Project PM relatively soon after certain things are confirmed, but this page will serve in the meantime as a repository for the information that's currently available. This effort is called Project PM. The major goals are to (a) discredit prominent pundits such as Thomas Friedman and Charles Krauthammer in order to reduce their negative influence on U.S. policy and (b) increase the positive influence of the more capable segments of the blogosphere. Both of these goals are to be pursued in part by way of the deliberate generation of critical mass among bloggers in such a manner as that segments of the traditional media will be prompted/forced to address certain critical issues as determined by a collective array of the most erudite and dependable bloggers in existence. A third goal that does not require the same critical mass or temporary control over the traditional media infrastructure involves the development of a communicational schematic that is superior to anything else in existence in terms providing bloggers with the best possible feed of raw information by which to produce content, as well as the best method by which readers can most easily find the best and most important of this content without having to sift through duplicate or sub-standard info, which is to say that it will discard some of the problems inherent to reddit, memeorandum, and other such sources as exist today (more on how this is to be accomplished may be seen below). All of this is to be achieved by way of Project PM network, which itself will make use of open-source software being designed for the purpose and developed by a fellow with a brilliant track record in innovative IT implementations. Obviously, this is not some magic bullet that will turn the media into an optimal system, but it will certainly have some positive effect, and to the extent that we can integrate more and more people into the network, this effect will of course be amplified. Meanwhile, others will no doubt adopt some of the methods we have developed for use in their own efforts. Perhaps most importantly, this will demonstrate to others that we are nowhere near fulfilling the positive potential of the internet as a medium, and get them thinking about the manner in which the internet provides us with the ability to provide for a degree of actionable collaboration among some of the world's most reasonable people in such a way that would have been impossible only a decade ago. Insomuch as that collaboration is the means by which human affairs are driven, such a grouping as makes use of intelligent, honest, and influential individuals and then gives them a means of communication and action that is even slightly superior to any other in existence would have unprecedented potential to achieve positive change, particularly by means of perpetuating information - information being, again, the fuel of human activity. Project PM Network Summary The institutions and structures that have developed over the past two decades of accelerating public internet use have had what we reasonably describe as a wholesome effect on information flow. But the information age is a work in progress, and thus there are potential improvements to be made. More importantly, there are improvements that can be made by an initially small number of influential participants working in coordination. The purpose of Project PM is to implement these solutions to the extent that participants are collectively able to do so, as well as to demonstrate the beneficial effects of these solutions to others that they might be spurred to recreate or even build upon them independently of our own efforts. The Problems Project PM is intended to address the following inefficiencies: (a) Watering down of contributor quality within participatory networks: Open institutions such as reddit.comtend to peak in terms of the erudition of the content conveyed a few years after coming about, with this being due to the particular dynamics of network growth. By definition, early users are early adapters, who themselves tend to be better-informed and otherwise relatively capable in terms of the value they bring to the network. To even know of such networks early in their existence is to pass a certain sort of test regarding the potential quality of one's contributions; as knowledge of the network expands, this "test" becomes easier, and to the extent that it does, the network is less "protected" from those who did not pass such a test by virtue of the fact that they did not know of the network until knowledge became more common. Obviously, failing to be aware of some particular institution does not come anywhere near precluding one from being intelligent and knowledgable in general and thus of value to the institution, but the influx of valuable participants versus damaging participants appears to decrease after a certain level of notoriety is reached. Again, the decline in the intellectual relevance of content at reddit.com is a good example of this. (b) Data overflow: The watering down process described above does not only result in one coming across information of relatively low quality, but also in having to contend with more of it. On reddit.com, for instance, a user who scans new submissions will find not only a certain amount of potentially useful information, but also some amount of almost certainly useless information. The watering down of contributor quality also contributes to the extent to which the latter is perpetuated within the network itself insomuch as that lesser contributors are more likely to vote up useless information, thus helping to ensure that the barriers built into the network in order to facilitate the viewing of important rather than unimportant content - in this case, a pre-established threshold of up votes necessary to bring something to the front page - will thereby lose their effectiveness. (c) Barriers to obtaining raw data: The obvious fact of data overflow - that some data is more useful than other data - is dealt with by means of selecting certain sources of information which one has identified as being a provider of quality output relative to other sources. Bloggers and others who require a steady stream of data in order to operate have certain methods of obtaining that data, and there is of course no reason to believe that any of these methods could not be improved upon to an extent that these improvements would be worth adapting. One has RSS feeds flowing from sources one has selected (and by virtue of having been selected, the sources must have been necessarily known to the blogger in the first place); one has algorithm-based sites like Memorandum.com (which merely shows what bloggers are talking about rather than necessarily providing any insight into what they should be talking about); one has democratic or pseudo-democratic sites such as reddit.com and digg.com; and one has the fundamentally one-way outlets of television and newspapers, the content of which is decided upon by a handful of producers or editors (who themselves are working within an incidental structure that does not appear to be of much value relative to what may now be found among the better portions of the blogosphere). A means of obtaining data that improves upon these and all other methods would be of great utility insomuch as that the quality of data is of course one major limiting factor with regards to the quality of output.. The Solutions By way of a network designed to take better advantage of the existing informational environment, Project PM can help to remedy the problems described above without significant effort on the part of participants, yet with potentially dramatic results on the efficiency of information flow. (a) Watering down of contributor quality within participatory networks: Project PM will greatly reduce the accumulation of low-value contributors by way of the method by which contributors are brought it. The network will be established with a handful of contributors who have been selected by virtue of intellectual honesty, proven expertise in certain topics, and journalistic competence in general. Each of these contributors has the option of inviting into the network any number of other bloggers, each of whom will initially be connected only to the contributor who brought him in. Each of these new participants also has the option of bringing others into the network in the same fashion as well as offering a connection to any other participant, as will anyone they bring in, and so on. To the extent that the original participants are of value in terms of their judgement, they may be expected to bring in participants of similarly high value, and so on; meanwhile, as the network expands, participants will be likely to form new direct connections to others whom they have determined to be of particular value relative to other participants, and conversely, to disestablish any direct connections they might have established to those whose output they find to be below par. Of course, none of this precludes the network from eventually encompassing participants of low desirability relative to that of the average participant, but to the extent that such a thing occurs, its effect are largely neutralized by way of the dynamic described below. (b) Data overflow: Information flows through the Project PM network by way of a single button accessible to each participant. When a participant either writes or receives a blog post or other informational element, the participant may "push" the item, thus sending it to all of those with whom he is directly connected in the network. In such a case as a participant pushes forward items that others may determine to be of little merit, the resulting clutter is only seen by the participant who brought such a low-value blogger into the network in the first place, as well as those whom the low-value blogger has to this point brought in himself along with those who have agreed to connect with him from elsewhere in the network. To the extent that a given participant exercises good judgment in establishing connections, then, he will only receive informational elements of value while also being able to quickly transmit them to contributors who will be able to make best use of such information. Meanwhile, below-average participants will have only very limited means by which to clutter the network, as informational elements become less likely to be pushed forward as they approach above-average participants within the network, who themselves are "buffered" from such things by way of the competent participants with whom they surround themselves by way of their connections and who, by virtue of their competence, are unlikely to push forward low-value information. (c) Barriers to obtaining raw data: The dynamics described in (a) and (b) collectively provide for a means of information inflow that should theoretically be superior to any other medium currently in existence in terms of overall quality, both by virtue of the network's improved organizational methods as well as the relatively high competence of participating bloggers relative to members of the traditional media outlets as a whole. Accessibility to particularly valuable items of information will be enhanced further by the option to set one's widget in such a way as to display any piece of information from the network, regardless of "proximity," if such information is pushed forward (which is to say, approved of other participants) a certain number of times. This should help to ensure that, as the network expands, particularly valuable information does not become unduly "regionalized." A variant on the widget for use by readers (as opposed to network participants) displaying information that meets similar thresholds of popularity within the network would likewise provide those readers with a source of information above and beyond other existing mediums.

While reading some of the commentary provided by you all here on Behind the Steel Curtain recently, I had several fresh thoughts about the linebacker situation for the Pittsburgh Steelers. There's not a whole lot to assess personnel wise at the outside linebacker position. 2008 Defensive Player of the Year James Harrison and third year sensation LaMarr Woodley have the two outside 'backer positions locked down for at least the next two or three years.

It's the inside linebacker position in Dick LeBeau's 3-4 defense that I've been contemplating since the Steelers brought back Larry Foote to Pittsburgh after his one year stay over in Detroit. 

The following is an excerpt from a very informative and well organized article about the Green Bay Packers 3-4 defense under recently hired defensive coordinator Dom Capers. In case you had forgotten, Capers was the architect of the Steelers defenses from 1992 until 1994 before being lured away by the Carolina Panthers to be the head coach for their inaugural season in 1995. Dick LeBeau was the defensive backs coach during Capers three years in Pittsburgh, so suffice it to say, much of what this article covers regarding Capers' 3-4 scheme in Green Bay is transferable to what the Steelers do defensively under Coach LeBeau.

The entire article is worth reading, but the excerpt I was compelled to share here is about the two inside linebacker positions in the 3-4 (bold emphasis mine):

There are two distinctly different ILB positions, the SILB and WILB, strong and weak. In Dom's system the SILB is named the buck, the WILB is named the mack. Unlike the OLB's, the ILB's do change sides based on which side the TE is on, with good reason. Usually if there is an uncovered gap or extra blocker, it is on the strong side. When looking at number counts, especially when a FB is involved, there are almost always as many blockers on the strong side as there are defenders. The buck and mack have very different skill sets as it relates to the run.

The buck is a banger. Remember one of the initial premises I made, a back with no blockers isn't going very far. The buck eliminates the blockers. If there is a G coming through, mix it up with the G and try to keep him in the gap. If the FB is coming through, stop him in his tracks. The last thing you want the buck doing is shedding the blocks, unless the RB is past or nearly past him. If he sheds the blocks early and fails to tackle the back, he made matters worse, now these blockers are in the secondary, the plus one has a blocker coming his way. Not good. The buck is supposed to engage blockers, not run around them.

If the buck has to take care of an offensive lineman, the best you can hope for is that he holds him up in the gap and doesn't get blown backward or thrown to the ground. Likewise he should hit the FB back and maintain control of the gap. The buck is essentially a small mobile defensive lineman, as it relates to the run, he almost always is part of the gap control scheme and has a gap assigned to him. He did his job if the back has to look elsewhere for an opening and if there are no blockers out in front of him.

The mack is the playmaker. He often has no gap assigned to him. He is to seek and destroy the guy with the ball. If the rest of the front did their jobs, he should be free of blockers. If not, he should shed any block immediately or go around them in pursuit of the ball. The mack is the star of the defense. He should always be around the ball. If he is blocked, the secondary has to make the play.

Okay, now, some of my thoughts about the Steelers inside linebacker situation. First though, some of the pertinent questions.

1. Why hasn't Lawrence Timmons developed into the monster that we all thought he would be by now heading into his fourth year?
2. How much does James Farrior have left in the tank at 34 years of age?
3. How will defensive coordinator Dick LeBeau utilize the trio of Farrior, Larry Foote and Timmons? (and really, I should say the quartet and add Keyaron Fox, as he's proven plenty capable of playing at a starting level when given the opportunity).

Though every nation is of some degree of contradictory nature if one looks hard enough, the United States offers ironies of particular significance. Here we have a nation that produces an incredibly large percentage of peer-reviewed papers relative to its population size, which produces a similarly large share of its patents, which attracts some of the best scientific minds in the world to some of the best scientific institutions in history, which has conceived and implemented such an incredible array of technological innovations that it would be impossible to calculate the degree to which our finest minds have contributed to the overall cause of mankind, its modern capabilities and its future aspirations. At the same time, it is a nation where science is under perpetual and sometimes successful assault, the great advances in our ability to understand and control evolution to the betterment of mankind co-existing along with a great population of people who benefit most from the resulting medical technology while denouncing the mindset that made such things possible. God often takes the credit that would more properly go to doctors and research scientists. The single greatest indictment of the American population as a whole may be seen in a survey of international attitudes towards the theory of evolution, itself among the most important and useful scientific constructs ever devised. Less than half of the adults of our republic ascribe to this most crucial bedrock of modern science. By this measurement [1], we are more akin to those relatively superstitious nations of Africa where the Enlightenment never took hold and those of the Middle East where it has been actively strangled by Islamic fundamentalism than to our First World friends in Europe and Asia. The American Christian conservative has a great deal in common with his Middle Eastern Muslim conservative counterpart. This explains why advocates of the fraudulent pseudo-theory known as intelligent design opted a few years back to hold a major conference in Turkey [2] - which, incidentally, is the only Western nation in which evolutionary science is regarded with more disdain than it is in the U.S. The United States, then, is the nation of the Enlightenment in which some half of the population longs for the dark ages - perhaps only unconsciously, but certainly without understanding the implications of what it is that they wish for, nor what will happen if they get it. There is a great deal at stake in the early 21st century, an era dangerously akin to earlier ones. But a cup half-empty is also a cup half-full. More to the point, it's a cup. One can always get a refill. Among those things that science has given us  is known as the internet. It is a shame that it is known as such - not because of any problem with the particular name, but rather because any term, and particularly one that encompasses so much at once, will over time accrue mental associations that may not be accurate in describing it or helpful in understanding it. Regardless of what we call it, it is certain that we do not understand it perfectly anymore than we are likely to fully understand anything, particularly something of great complexity. It is not much of a task to encompass in one's mind an apple, to appreciate the positive and negative aspects of such a thing, to model in our thoughts the possibilities inherent to an apple, to be reasonably certain that we have taken best advantage of what the apple has to offer us and what it could potentially do for the civilization in which we each have a stake. The apple is very limited, after all; it lacks relative complexity. The internet, by contrast, is essentially unlimited, and as complex as anything we have yet encountered other than ourselves. In terms of our ability to predict what will come of it, it is the very opposite of an apple. And just as we probably need not stop and re-evaluate the apple from time to time in search of some revolutionary use by which we might perhaps solve some great problem, we would be negligent in our humanity were we not to give the internet some great deal of consideration, to take a moment or two to ensure that we are taking best advantage of this phenomenon that is unlike anything else. The internet is still in its infancy, and the possibilities inherent to the medium have yet to be fully explored. New uses and dynamics keep popping up. They pop up in such quick succession that we do not even wonder at them anymore, would find it surprising only if new surprises did not appear. Recently there arose a new surprise whereby one may click a button and find one's self face to face with some random person in some random place, whether at the other end of the globe or in one's own city. This development was received with the same high degree of frivolity as one would expect from a society in which frivolity is often confused with maturity and worldliness; a breezy tone flows through most written accounts one finds on the subject, the inevitable encounters with male genitalia are emphasized to an extent far greater than their actual sociological implications, and one need not bother viewing any television news segments to know that whatever significance may exist here will not survive the dynamics of the medium. Meanwhile, for the first time in our two million year history, it is now a simple matter to communicate both visually and audibly with some individual with whom we were last connected perhaps a hundred thousand years ago by way of some common ancestor, and whom the twin barriers of physical geography and political reality would have successfully kept out of our reach were it not for the internet and the era that it has come to define. Again, though, this is merely another wonder in an age of wonders, one in which it is drastically simple for an individual to have some profound effect on the world and its workings. Of greater fundamental importance than any particular development is the fact that such wonders now come about at such a pace and with such ease; this should tell us that there are additional wonders to be instigated, and that changing the habits of a hundred million people is simply a matter of identifying an opportunity and then acting on it. Institutions that have existed for tens, hundreds, or thousands of years are now exceedingly vulnerable to reform and even overthrow. Such institutions were once safe by virtue of having developed in a certain environment. That environment is gone forever. I have a new monthly column for The Skeptical Inquirer, the admirable old magazine put out by the similarly admirable Committee For Skeptical Inquiry [3]. Skepticism, incidentally, is more than the practice of dismissing ghosts or holistic medicine; it is the practice of distinguishing between that which we know and that which we think we know. Some great degree of human misery has been the result of mistaken beliefs, either of the sort that prompts us to stop searching for the truth or the sort that prompts us to actively do harm to others. The plagues that killed off sizable portions of Europe on several occasions could have been drastically minimized if humanity had applied skepticism to their beliefs rather than assuming that they knew the causes of such diseases; the various religious wars that defined the monotheistic movements could have been similarly minimized in the absence of such a fine recruiting tool as God's Seal of Approval. Skepticism is not simply the hobby of killjoys; it is a matter of life and death. If the reader finds such an assertion to be outlandish, then the reader almost certainly lives in a society in which skepticism has made relatively grand strides, rather than in some country in Africa where myths regarding HIV and AIDS have contributed to the illness and deaths of millions, where virgin girls are routinely raped in a misguided and immoral effort to cure a disease that thrives in environments marked by credulity. The AIDS-related rapes and religion-related witch burnings  may seem alien to those among among our citizens who consider themselves sophisticated members of  modern society - people who ascribe to the Bible, horoscopes, the power of prayer, the concept that God will intervene in one's financial life or will provide some degree of support to our military expeditions by virtue of His keenness on our form of government. But there is no fundamental difference between the potion-brewing shaman and the bestseller-composing Rick Warren, as both deal in magic and encourage others to depend on it in matters of life and life and death. Both are responsible for the consequences that result from the dependency of their followers on magic over hard work. Neither take responsibility for any of these consequences, though they are more than happy to take one's money. The real difference between the two is that the witch doctor is highly limited in his means of reaching new customers, whereas Warren is fortunate enough to live in a society in which materialist science allows him to fly to presidential inaugurations in planes, address new marks via television and radio, and otherwise make use of the products of secular thinking in the process of denouncing such a world view as has made it possible for him to be heard in the first place. More fundamentally, the difference is that Warren lives in a world built by his ideological enemies, whereas the witch doctor is at home among others sharing his own mindset. Today, the internet provides a new means by which Warren and other mystics of the sort may ply their trade to an extent that would have been impossible not long ago. Does this mean that the internet will have an overall negative effect on the wholesale rationality of mankind?  My first Skeptical Inquirer column [4] makes the oddly controversial case that the cause of skepticism is in fact well-served by the communications age despite the challenges it brings. More importantly, I will also be making the case - in fact, will be spending the rest of my professional life doing so - that the dynamics of the internet provide us with an opportunity to redefine the world around us in a manner that would have been impossible just a decade ago, and that this is only a matter of a few individuals taking responsibility in accordance with their abilities. As I noted above, we would be negligent in our humanity were we not to give the internet some great deal of consideration, to take a moment or two to ensure that we are taking best advantage of this phenomenon. Information is fundamental to the manner in which human society functions, and insomuch as that human society has rarely functioned as well as we know that it can as proven by instances in which it has functioned relatively well, we may determine that aspects of society may be drastically improved, and that such improvements may be spurred merely by an improvement in the flow of information. I have devised a schematic whereby information may flow in a manner superior to anything that has existed before, and which may thus be expected to have a significant positive effect to the extent that the schematic is adopted by others who operate in the realm of information. Certain individuals more prominent and resourceful than myself have already agreed to perpetuate the plan either by adopting it, promoting it, or supporting it, and thus the plan is already a success to the extent that a rock atop a clip has locked into it a certain amount of potential energy. More are in the process of joining, which is to say that, to some degree and at some pace, the plan will go into effect. But with more participants, the plan will achieve greater and quicker success. The details will be announced soon, but in the meantime I am in search of additional participants to serve in a capacity crucial to the exact manner in which things proceed. A more formal announcement is forthcoming, at which point I will be recruiting in earnest, but if you are an individual who feels sufficient responsibility for the world around you that are willing to give a few minutes of your life in determining whether or not my plan is of sufficient viability to merit your backing, I would ask you to contact me at barriticus@gmail.com, in which case I will provide you with more information and answer any questions you may have. In the meantime, I would simply note that never has there existed such opportunity for revolution in human affairs. I am merely proposing that we take advantage of this opportunity. [5] [1] http://www.nytimes.com/imagepages/2006/08/14/science/sciencespecial2/20050815_EVO_GRAPHIC.html [2] http://www.evolutionnews.org/2007/03/id_conference_a_success_in_tur.html [3] http://www.csicop.org/ [4] http://www.csicop.org/specialarticles/show/internet_and_the_republic_of_skepticism_part_one [5] http://trueslant.com/barrettbrown/files/2010/03/All-Info-All-Ways1.png

A common needed and used feature in Silverlight is the Fullscreen Mode. A feature to switch the current application from a hosted in browser to fullscreen mode. This features is interesting in scenarios like Point of Sales- or Games-Applications.

Normal Browser Mode

Full Screen Mode

Livepreview here

XAML-Code

<UserControl x:Class="FullScreenBehavior.MainPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d"
    xmlns:TheOliver_Controls="clr-namespace:TheOliver.Controls"
    xmlns:i="clr-namespace:System.Windows.Interactivity;assembly=System.Windows.Interactivity"
    d:DesignHeight="300" d:DesignWidth="400">

    <Grid x:Name="LayoutRoot" Background="White">
        <Button
            Content="Button"
            Height="23"
            HorizontalAlignment="Left"
            Margin="10,10,0,0"
            Name="button1"
            VerticalAlignment="Top"
            Width="75">
            <i:Interaction.Triggers>
                <i:EventTrigger EventName="Click">
                    <TheOliver_Controls:FullScreenAction/>
                </i:EventTrigger>
            </i:Interaction.Triggers>
            
        </Button>
        <Rectangle Stroke="Black" Margin="155,113,141,110">
            <i:Interaction.Triggers>
                <i:EventTrigger EventName="MouseLeftButtonUp">
                    <TheOliver_Controls:FullScreenAction/>
                </i:EventTrigger>
            </i:Interaction.Triggers>
            <Rectangle.Fill>
                <LinearGradientBrush EndPoint="0.5,1" StartPoint="0.5,0">
                    <GradientStop Color="Black" Offset="0"/>
                    <GradientStop Color="White" Offset="1"/>
                </LinearGradientBrush>
            </Rectangle.Fill>
        </Rectangle>
    </Grid>
</UserControl>

Sourcecode

// Copyright © Microsoft Corporation.  All Rights Reserved.
// This code released under the terms of the 
// Microsoft Public License (MS-PL, http://opensource.org/licenses/ms-pl.html.)

using System.Windows;
using System.Windows.Interactivity;

namespace TheOliver.Controls
{
    public class FullScreenAction : TargetedTriggerAction<FrameworkElement>
    {
        protected override void Invoke(object o)
        {
            Application.Current.Host.Content.IsFullScreen =
                !Application.Current.Host.Content.IsFullScreen;
        }

    }
}

 

Solution download

Database Designer & Administrator for 3vTV
We are seeking to fill this position immediately. This position will be a contract position with the candidate working remotely located any where throughout the U.S. Must have Android as we are building Android based applications.

This position will be the Lead programmer and work side by side with other developers.

We're currently sourcing for a Part-Time Senior, SQL Server DBA/Developer
with 2+ years of professional experience to join us and work remotely.
Candidate can be located any where throughout the U.S.

This position is for a SQL Server DBA/Developer with knowledge of
designing, developing, managing, and troubleshooting production and
development databases and maintaining data integrity standards.

The ideal candidate will possess a solid foundation in database design
and and administration with mySQL. It is equally important that the
successful candidate demonstrates great attention to detail and is
able to meet deadlines consistently and efficiently. Preferred
candidates will possess a Bachelor*s degree in Computer Science,
Information Technology, or a related field.

Required Skills:

* 2+ years of database experience including any SQL database
* Experienced in database design, utilizing design tools. Must have experience creating relational schemas.
* Experience administering database servers including installation,
configuration, maintenance, backups, and replication, etc*
* Expert in writing SQL, stored procedures, triggers, packages and
user-defined functions.
* You should have excellent communication and collaboration skills, be
an eager, go getter with a great can do attitude and a hunger for
knowledge.
* Strong XML experience

Desired Skills:
* Experienced with mySQL
* Linux shell scripting
* Development skills in JAVA will be a plus.


Qualified candidates please forward resume to JohnLafser@3vTV.com along with short cover letter with links to previous projects and achievements.

Are you floored by the recent Supreme Court decision unleashing corporate contributions to political campaigns?

Have you always suspected that voting in Congress is influenced by the contributions of interest groups, but never had the data to prove it?

Does a chance to work magic with that data to develop sophisticated tools to support high-profile research fill you with glee?

MAPLight.org, a cutting-edge non-partisan research organization located in Berkeley California, is looking for the right stuff in a PHP/MySQL developer. We need someone who speaks in SQL and dreams in PHP. Someone who can look at gigabytes of data and quickly develop innovative strategies for getting the most out of our research and analysis tools.

Responsibilities:
* Work in a small, dynamic team to develop new web-based tools to support a talented research team and an audience of millions.
* Help to develop and optimize database schema and queries.
* Develop and improve processes to glean data from a wide range of data sources that is the backbone of our research.
* Enhance our Web 2.0 applications with top-notch PHP coding and client-side scripting.

Qualifications:
* 5+ years web development, with a strong focus on LAMP and open source tools.
* 5+ years database experience in MySQL or Postgres.
* Very strong SQL skills.
* Expertise with PHP, DHTML/CSS/Javascript, XML.
* Knowledge of Drupal development.

A knowledge of the American political system (or a desire to learn) is preferred, as is a passion for openness and transparency.


This is a full-time position with salary and benefits commensurate with experience -- we are not interested in a contract situation at this time. MAPLight.org is located in downtown Berkeley, California (close to BART). Telecommuting is an option for exceptional candidates, but be prepared to put some effort into proving that are the right person for that option.

Interested candidates should submit a cover letter and resume to eric [at] maplight [dot] org.

PLEASE INCLUDE A CODE SAMPLE AND URLS TO YOUR ONLINE WORK


About MAPLight.org MAPLight.org, a groundbreaking public database, illuminates the connection between campaign donations and legislative votes in unprecedented ways. Elected officials collect large sums of money to run their campaigns, and they often pay back campaign contributors with special access and favorable laws. This common practice is contrary to the public interest, yet legal. MAPLight.org makes money/vote connections transparent, to help citizens hold their legislators accountable.

MAPLight.org’s small, effective team has won numerous awards, including a 2008 Webby award nomination for best Politics website. Learn more at http://www.maplight.org.

Gordon Humphrey writes: As past-editor of Woodworking Magazine (which I am more than a little sorry to see disappear), and still editor of Popular Woodworking Magazine and designer of the sideboard on the cover of the Summer 2009 issue of Woodworking Magazine (the featured project of that issue), I wish to call you attention to what I believe to be an error in the cutlist on page 12. The web frame stiles are said to be 33" long. Given that the interior measurement of the table is 34" and you state (page 15) that the "guts" should be "a little too tight" with respect to the interior of the base, this listing of 33" should appropriately be more like 34", preferably 35", or so.

As a "intermediate beginner" woodworker, I jumped ahead and made the "guts" before reading your statement on page 15: "Measure the inside of your base and make the web frame so it fills that space exactly,...." This sentence may serve to absolve you of responsibility, I suppose, but I don't see why a cut list should be wrong. If it's wrong, what's the point of it? I wasted lots of time making the mortises and tenons for the "guts" and assembling the "guts" before gluing up the base (and therefore before measuring or looking at the 34" interior dimension shown in the schematic. That was time and effort wasted. I don't understand why that should have happened, since I don't particularly like making wasted mortises and tenons with wasted wood. (Does this disqualify my from the realm of woodworkers, I wonder?).

I shall take 20 percent of the blame and let it serve as a lesson to me, but I suggest that as editor you need an editor or a proofreader or a reader stand-in who builds your stuff and checks your figures before going to press.

— Gordon Humphrey

Editor Christopher Schwarz responds: Sorry for the error. We don’t make too many of those and we do have proofreaders. However, this is a human endeavor, and we make mistakes.

I will say this (and I’ve said this before), that woodworkers should never (and I rarely use that word in woodworking) cut out all the pieces from any cutlist. Even their own. Even from a cutlist that has been checked 1,000 times.

The proper procedure is to build the exterior case and base all your measurements off that as you proceed. Heck, I think that providing parts lists for drawers is a waste of valuable ink. But when we remove them, we get phone calls.

In any case, I hope you can find a use for that extra wood.

— Christopher Schwarz

Ian McEwan excels at climate science but his one-dimensional protagonist makes you shudder

Solar is a sly, sardonic novel about a dislikable English physicist and philanderer named Michael Beard. He's a recognisable Ian McEwan type, a one-dimensional, self-deceiving man of science. We have met others like him before in McEwan's novels – such as Joe Rose, the science writer who narrates Enduring Love, or Henry Perowne, the brian surgeon protagonist of Saturday – but none is quite as repulsive as Beard. Perhaps McEwan should have written against expectation by choosing as his protagonist a scientist who has a profound artistic sensibility in the model of his friend Richard Dawkins, or an artist who is articulate in the language of science, as McEwan is himself. As it is, he remains a determined binarist; what continues to interest him are stark dichotomies, the clash and interplay of stable oppositions. Repeatedly in his fiction he sets reason against unreason, science against art, the mind against the body, technology against nature.

Beard, who we are encouraged to believe won a Nobel prize in physics as a young man for something called the Beard-Einstein Conflation, is a short, fat, balding, much-married man of immense bodily appetites and scant self-discipline. He rapaciously consumes food, women and drink, with little regard for the consequences. He's a resolute short-termist, fearful of commitment and of becoming a father, living for the here and now. His behaviour is a local example of the more general problem of human over-consumption: just as Beard devours everything around him, so we are devouring our world, with its finite resources and fragile ecosystems.

The trick of the novel, its central comic turn, is to make Beard, the greedy, selfish uber-consumer, an accidental expert on anthropogenic climate change. Through his expertise as a physicist, and then his opportunism in stealing the research ideas of a graduate student who works with him at an institute in Berkshire known only as the Centre, Beard is engaged in a programme to create cheap renewable energy through a process of artificial photosynthesis (you'll need to read the book to be filled in on the science).

McEwan's great gamble is to narrate Solar, which is in three parts and spans nine years, from 2000 to 2009, entirely from Beard's point of view. Some of this is satisfying, especially the pithy scientific elaborations: McEwan, who has a precise, technician's vocabulary, has swotted up to PhD level on physics, just as he did on neurosurgery for Saturday, musicology for Amsterdam and molecular biology for Enduring Love. None of this extracurricular learning feels perfunctory, especially when compared with, say, a novel such as Martin Amis's The Information. In that novel, disquisitions on infinity, black holes, dwarf planets and astronomy felt imposed on the narrative rather than being intrinsic to it. In Solar, the physics never feels forced or unearned but rather is embedded in the deep structures of Beard's consciousness. We see the world just as he does, in all its cold reductiveness.

In McEwan's early fiction, in his strange, experimental short stories and novellas, with their isolated, sexually deviant male protagonists, he wrote from the outside in, as it were. His was always the controlling intelligence, aggressively masculine, and he followed his young male protagonists less in thought than in action, detailing their psychosis and alienation with the cold detachment of a coroner examining a corpse. So devoid of emotion were these early fictions that reading them felt almost like an act of voyeurism. McEwan appeared to have no feelings for his characters; instead, he dissected them as if they were rats in a laboratory experiment. Here was, as John Updike said of Amis in a different context, "an atrocity-minded author" – but one with perhaps the most controlled, exact and fastidious prose style of all contemporary British novelists.

McEwan's cruellest book, the one in which the violence seems most gratuitous and nasty, is The Comfort of Strangers (1981), his novella about a young British couple adrift in an autumnal Venice of shadows and fear that marked a point of transition for him: after this, and a long period of silence, he returned as a different writer. The instinct for cruelty remained but it was mitigated by a much deeper, more sophisticated, even feminine, moral and aesthetic sensibility. From The Child in Time (1987) onwards, he was less a postmodernist than a realist, with a 19th-century interest in character, agency and storytelling, in the what, how and why of the human dilemma. In addition, he was developing a modernist's concern with consciousness, and began to experiment with different ways of representing the essence of what Virginia Woolf called the "quick of the mind".

McEwan was now writing from inside out: thought determined action, and there was a new descriptive density to his writing. Where once he had shrunk and compressed, he now expanded and inflated. He was no longer a miniaturist, and yet he was still a writer of great, self-contained set-pieces, such as his description of a ballooning accident that provides the celebrated opening to Enduring Love; or the disappearance of an infant from a supermarket at the beginning of The Child in Time, a scene of such intense and urgent panic that it would torment any parent who read it; or the extended scene in On Chesil Beach, where the virginal newlyweds try and fail to consummate their marriage, with devastating consequences for the rest of their lives.

Yet what unifies all McEwan's fiction is his preoccupation with the randomness of human endeavour in a post-religious, Darwinian world: his novels invariably turn on one sudden, unaccountable, life-changing happening or fatal equivocation. In Solar, the moment of crisis occurs when Beard returns from a trip to the Arctic to discover one of his graduate students at play in his house. It's quickly apparent that he is having an affair with Beard's unhappy and desperate wife (the marriage is over; Beard is a serial adulterer), and, in the ensuing confusion, the student falls and hits his head: "No breathing, no pulse." Instead of helping the student or calling for an ambulance, Beard stumblingly intervenes to make it appear as if he has been murdered – as you would – so as to frame one of his wife's previous lovers, a thuggish builder whom Beard has confronted earlier in the novel.

The circumstances of the death are used by Beard to extricate himself from his marriage, to punish the loathed builder and to reanimate his moribund career by appropriating the student's research into climate change and claiming it as his own. In one bound, it seems, he's free.

That's a lot of change to believe in, however, and from this point everything feels excessively neat, ruthlessly schematic. Solar is very similar in style to the Booker prize-winning Amsterdam, especially in its narrative tidiness, jauntiness of tone and desire to punish foolish men. But Amsterdam was a novella, whereas Solar feels as if it has been stretched far beyond its natural length. Much of the first part, which is set in 2000 and culminates in the death of the student, reads like an exercise in extended scene-setting, to no obvious purpose or effect. The protracted episode in which Beard travels with a group of scientists, artists and green activists to the Arctic, played mostly for laughs – he pisses outside and his penis freezes; he is menaced by a polar bear – is laboriously over-described. There is, sentence by sentence, an uncharacteristic verbosity.

The chief revelation of the Arctic mission is to show how the bootroom, where the well-intentioned group keeps its foul-weather clothing, becomes, after only a few days, a site of anarchy and chaos. This then is another parable of human rapacity: we take what we can, when we can.

The best and most complex scene occurs towards the end of the second part. Beard and his latest lover, a kind, generous, full-figured woman, are together at her house. She has prepared a meal, and just before they sit down to eat it she tells Beard that she's pregnant and determined to keep the baby. In sentences of extraordinary poise and precision, McEwan contrasts Beard's selfish sense of revulsion at the news with that of his lover's hopeful joy. So minutely does he track Beard's ever-shifting, contradictory positions you have a returning sense, so familiar in McEwan's fiction, of events somehow sliding inexorably towards disaster. But then the lens blurs, there's an unexpected softening of focus and, after having sex, exhausted, the couple fall asleep.

We next encounter Beard, four years later, in 2009, at the beginning of part three. He's in New Mexico preparing for a conference, and it is there, under the ferocious sun, that a lifetime of carelessness eventually catches up with him in a denouement that wouldn't be out of place in a West End farce.

It was always going to be high risk, wagering so much on having as your central character a comic grotesque so loathsome and self-pitying, with thoughts mostly so banal, and then leaving the reader trapped, unrelieved, in his company for nearly 300 pages.

In Atonement, the character Bryony writes fiction in which she seeks "to show separate minds, as alive as her own, struggling with the idea that other minds were equally alive".

What is absent from Solar, ultimately, are other minds, the sense that people other than Beard are present, equally alive, with something to contribute. Without them, after a while, it feels as if you are locked inside an echo chamber, listening only to the reverberations of the one same sound – the groan of a fat, selfish man in late middle age eating himself.

Jason Cowley is editor of the New Statesman

In the first two parts in this series we've seen how to create a custom provider, do claims augmentation and register the provider, as well as how to add a hierarchy to the people picker.  In this post we'll talk about how to implement searching for our claims in the people picker with our custom provider.  And this is really where the rubber meets the road.  Without having a way to select our custom claim, we can't provision permissions in the site based on who your favorite basketball team is.

First, I want to take a moment to really let that sink in, because this is an important yet not really well understood point.  By provisioning permissions based on an attribute of the user, rather than who the user is, it opens ALL sorts of possibilities.  It can mean, as it does in this scenario, that I don't care who you are.  I don't care how you logged in.  I don't care if you're a Windows user or an FBA user.  All I care about is who your favorite basketball team is.  Here's a classic problem from SharePoint 2007 that still exists in SharePoint 2010 - when I'm at work I log into my SharePoint site using my Windows credentials - Steve Peschka.  But when I go home I hit the site via our extranet and I have to use forms based authentication.  For our scenario, assume that I am FBA user "user1".  What's the problem?  Well the issue is that to SharePoint these are two different users.  There isn't a way in SharePoint 2007 or SharePoint 2010 to do some kind of user mapping, where we say 'well, the speschka Windows user and user1 FBA user are really the same person'.  Guess what?  We no longer have to care.  Why don't we?  Because we aren't assigning permissions on who the person is, we're assigning permissions based on what their favorite basketball team is.  The scenario I've implemented is simplified, but you could imagine a piece of corporate metadata that is associated with a person, and the claims provider doing a lookup to some other system to figure out what all the different identities are a person uses - Windows, FBA, PeopleSoft, SAP, CRM, etc. - and being able to map some other identifier or set of claims to that identity.  It's then those claims that are used to grant access to resources! 

You want single sign on between your different web applications?  Again, user1 in webapp1 is not the same as user1 in webapp2.  Who cares?  As long as I'm using my favorite basketball team claim that person can move seamlessly between those web applications because we are augmenting it each time they authenticate.  I don't want to tell you it's perfect because it's not, but it sure is a lot closer to it than previous security models.

Okay, while that sinks in let's start talking about supporting search in the people picker.  In order for our custom claims provider to support search we need to add support for the following properties and methods:  SupportsSearch, SupportsResolve, FillSearch, FillResolve (the overload with SPClaim as an input parameter),  FillSchema and FillEntityTypes.

The properties should be pretty straightforward at this point so let's look at them first:

public override bool SupportsSearch

{

      get

      {

            return true;

      }

}

 

public override bool SupportsResolve

{

      get

      {

            return true;

      }

}

 

FillSearch is probably going to strike you as the most interesting, as it should.  So let's look at that code:

protected override void FillSearch(Uri context, string[] entityTypes,

      string searchPattern, string hierarchyNodeID, int maxCount,

      Microsoft.SharePoint.WebControls.SPProviderHierarchyTree searchTree)

{

 

//counter to track what node we're in; will be used to call into

//our helper arrays that were covered in parts 1 and 2 of this series

int teamNode = -1;

 

//NOTE:  If we weren't using hierarchies in the picker, then we would

//probably just use a List of PickerEntity instances so that we could

//add them all to the people picker in one call.  I've stubbed it out

//here just for illustration purposes so you could see how you

//would do it

//picker entities we'll add if we find something

//List<PickerEntity> matches = new List<PickerEntity>();

 

//node where we'll stick our matches

Microsoft.SharePoint.WebControls.SPProviderHierarchyNode matchNode = null;

 

//look to see if the value that is typed in matches any of our teams

foreach (string team in ourTeams)

{

      //increment team node tracker

      teamNode += 1;

 

      //simple way to do a string comparison to the search criteria

      //this way all a person has to type in to find Blazers is "b"

      if (team.ToLower().StartsWith(searchPattern.ToLower()))

      {

            //we have a match, create a matching entity

            //this is a helper method that I'll explain later

            PickerEntity pe = GetPickerEntity(team);

 

            //if we didn't have a hierarchy we would add it here

            //using the list described above

            //matches.Add(pe);

 

            //add the team node where it should be displayed; make

            //sure we haven't already added a node to the tree

            //for this team's location

            if (!searchTree.HasChild(teamKeys[teamNode]))

            {

                  //create the node so we can show our match in there too

                  matchNode = new

                  SPProviderHierarchyNode(SqlClaims.ProviderDisplayName,

                  teamLabels[teamNode],

                  teamKeys[teamNode],

                  true);

 

                  //add it to the tree

                  searchTree.AddChild(matchNode);

             }

             else

                  //get the node for this team

                  matchNode = searchTree.Children.Where(theNode =>

                  theNode.HierarchyNodeID == teamKeys[teamNode]).First();

 

            //add the picker entity to our treenode

            matchNode.AddEntity(pe);

      }

}

 

//if we were adding all the matches at once, i.e. we weren't using

//a hierarchy, then we would add the list of matches here

//if (matches.Count > 0)

//    searchTree.AddEntities(matches);

}

 

So again, the code is probably pretty self-explanatory.  Basically what's happening is the user has typed something in the search box in people picker and click the search button.  Our provider is called since our SupportsSearch property returns true.  The method that is called is the FillSearch method.  In there we look at what the user typed in, which is provided to us with the searchPattern input parameter.  We look at all of our team names and check to see if any of them start with the value that was typed in the search box.  If it does, we create a new PickerEntity, we either find or create a search tree hierarchy node with the location of that team, and we add our PickerEntity to that node.

Now the code above shows us using a special helper function called GetPickerEntity where were pass in the name of the team we found.  Let's look at that code:

private PickerEntity GetPickerEntity(string ClaimValue)

{

 

//use the helper function!!

PickerEntity pe = CreatePickerEntity();

 

//set the claim associated with this match

pe.Claim = CreateClaim(SqlClaimType, ClaimValue, SqlClaimValueType);

 

//set the tooltip that is displayed when you hover over the resolved claim

pe.Description = SqlClaims.ProviderDisplayName + ":" + ClaimValue;

 

//set the text we'll display

pe.DisplayText = ClaimValue;

 

//store it here too in the hashtable **

pe.EntityData[PeopleEditorEntityDataKeys.DisplayName] = ClaimValue;

 

//we'll plug this in as a role type entity

pe.EntityType = SPClaimEntityTypes.FormsRole;

 

//flag the entry as being resolved

pe.IsResolved = true;

 

//this is the first part of the description that shows

//up above the matches, like Role: Forms Auth when

//you do an FBA search and find a matching role

pe.EntityGroupName = "Favorite Team";

 

return pe;

}

 

In a nutshell what's happening here is we're creating a new PickerEntity.  As the code comments indicate, the CreatePickerEntity is another helper function that SharePoint provides.  I strongly recommend again that you use the helper functions whenever you can.  So after we create our PickerEntity we need some way to describe what it "is".  In this case, "it" is a claim for favorite basketball team.  The way we describe that to the system is to create a new claim, using the same approach we did for our claims augmentation code.  This is why we passed in the team name to this helper function, so it could stick the person's favorite team into our favorite team claim.  After that we just set a number of properties on the PickerEntity so that it displays and behaves like other claims you see for Windows claims or FBA claims.  When we're all done we return the PickerEntity that we created.

We're still not quite done however.  If we stopped at this point, we would see the claim in the picker and be able to select it, but when we clicked the OK button to close the people picker dialog, what would happen is that you would see the claim in the type-in control, but it would show up with the red-squiggly underline indicating that it is not resolved.  And try as you might, you wouldn't be able to resolve it at that point.  What actually happens under the covers is when you click the OK button on the picker dialog to close it, SharePoint tries making another call into your provider, into the FillResolve method.  This is an overloaded method; the one that is called into when you close the picker dialog is the one that contains an SPClaim input parameter.  So let's take a look at the implementation of that method:

protected override void FillResolve(Uri context, string[] entityTypes,

      SPClaim resolveInput,

      List<Microsoft.SharePoint.WebControls.PickerEntity> resolved)

{

 

//same sort of code as in search to validate we have a match

foreach (string team in ourTeams)

{

if (team.ToLower() == resolveInput.Value.ToLower())

{

      //we have a match, create a matching entity with helper method

      PickerEntity pe = GetPickerEntity(team);

 

      //add it to the return list of picker entries

      resolved.Add(pe);

}

}

 

So this method is pretty straightforward.  You may remember from part 1 of this series that the SPClaim's Value property is going to contain the value for the claim.  Since there may have been multiple items selected in the people picker, as we're passed in a claim we enumerate through our team list again to see if the claim value matches one of our teams.  If it does, we call our helper method to create a new PickerEntity, and then we add it to the list of PickerEntity instances for SharePoint.  Now we see a resolved entry in the type-in control after our selection in the people picker. 

There are two other methods you need to implement, but you really don't need to provide any code in them.  They are the FillSchema and FillEntityTypes methods.  All I've done in my implementation is just comment out throw exception boiler plate code that was inserted when Visual Studio implemented the class methods for me:

protected override void

FillSchema(Microsoft.SharePoint.WebControls.SPProviderSchema schema)

{

//throw new NotImplementedException();

}

 

protected override void FillEntityTypes(List<string> entityTypes)

{

            //throw new NotImplementedException();

}

 

To illustrate, here's what people picker looks like after we've done a search for "DVK":

And here's what the type-in control looks like after we've added our claim and clicked the OK button:

So from a code perspective, we are complete, but it's time to ties this back to the discussion at the start of the post...now we can provision permissions based on who your favorite basketball team is.  In this case, I've decided that people whose favorite team is DVK Jovenut should be able to contribute content to the site.  In this case, I just added this claim to the Members group.  Here's how that looks ("Windows Claims" is the name of the site collection; don't let that throw you):

Furthermore, remember what I said about I don't care anymore who you are or how you logged in.  By way of example, I have NOT granted ANY Windows user, Windows group, FBA user or FBA role rights anywhere in the site.  I've ONLY added my claim for DVK Jovenut to the Members group, and I also added my claim for Blazers to the Visitors group.  By way of proof, here's what my list of all groups looks like for the site collection:

Hopefully this will really get you thinking about exactly how powerful claims authentication and claims providers can be.  In the final post in this series, I'll show you how to implement support for people typing in a claim name in the type-in control.

The DDL code paths for Drizzle are increasingly different from MySQL. For example, the embedded_innodb StorageEngine CREATE TABLE code path is completely different than what it would have to be for MySQL. This is because of a number of reasons, the primary one being that Drizzle uses a protobuf message to describe the table format instead of several data structures and a FRM file. We are pretty close to having the table protobuf message format being final (there’s a few bits left to clean up, but expect them done Real Soon Now (TM)). You can see the definition (which is pretty simple to follow) in drizzled/message/table.proto. Also check out my series of blog posts on the table message (more posts coming, I promise!). Drizzle allows either your StorageEngine or the Drizzle kernel to take care of storage of table metadata. You tell the Drizzle kernel that your engine will take care of metadata itself by specifying HTON_HAS_DATA_DICTIONARY to the StorageEngine constructor. If you don’t specify HTON_HAS_DATA_DICTIONARY, the Drizzle kernel stores the serialized Table protobuf message in a “table_name.dfe” file in a directory named after the database. If you have specified that you have a data dictionary, you’ll also have to implement some other methods in your StorageEngine. We’ll cover these in a later post. If you ever dealt with creating a table in MySQL, you may recognize this method: virtual int create(const char *name, TABLE *form, HA_CREATE_INFO *info)=0; This is not how we do things in Drizzle. We now have this function in StorageEngine that you have to implement: int doCreateTable(Session* session, const char *path,                   Table& table_obj,                  drizzled::message::Table& table_message) The existence of the Table parameter is largely historic and at some point will go away. In the Embedded InnoDB engine, we don’t use the Table parameter at all. Shortly we’ll also get rid of the path parameter, instead having the table schema in the Table message and helper functions to construct path names. Methods name “doFoo” (such as doCreateTable) mean that there is a method named foo() (such as createTable()) in the base class. It does some base work (such as making sure the table_message is filled out and handling any errors) while the “real” work is done by your StorageEngine in the doCreateTable() method. The Embedded InnoDB engine goes through the table message and constructs a data structure for the Embedded InnoDB library to create a table. The ARCHIVE storage engine is much simpler, and it pretty much just creates the header of the ARZ file, mostly ignoring the format of the table. The best bet is to look at the code from one of these engines, depending on what type of engine you’re working on. This code, along with the table message definition should be more than enough

Whenever we process information, interpret an experience, or organize our actions, we do so within the context of existing intellectual knowledge abstractions—schemas. Schema theory has been developed within the field of psychology in an attempt to explain how these cognitive knowledge structures are derived from personal experience and how they are organized in memory. Furthermore, schema theory has allowed us to investigate how these schemas serve as prototypes in memory and how they influence our interpretations of events.

As an example, we can briefly examine a little story discussed in the 1985-published proceedings of Nobel Conference XX, How We Know (edited by Michael Shafto): “John went to a restaurant. He ordered lobster. He left a small tip. He left.” As we consider the story, we might conclude that we actually know quite a bit about John even though explicit items of information are not mentioned in the passage. We might say that we know that John ate lobster, was served by a waitress or a waiter, and was not pleased with the service and / or the food. What this brief example suggests is that when we process information, we do so within the context of our present cognitive knowledge. Therefore, when I read a paragraph (Sunday’s gospel passage, for example), or experience an event (the sacrament of Reconciliation), or interact with others (a social in the parish hall), I process and interpret these situations within the context of what I know about the world—that is, within the context of my stored schemas.

To further clarify this important point about how we process information, let’s look at a second example. I referenced the following paragraph years ago, in my dissertation research into the effects of contextual information on the comprehension and memory of prose material:

With hocked gems financing him / Our hero bravely defied all scornful laughter / That tried to prevent his scheme / Your eyes deceive / He had said / An egg / Not a table / Correctly typifies this unexplored planet / Now three sturdy sisters sought proof / Forging along sometimes through calm vastness / Yet more often over turbulent peaks and valleys / Days became weeks / As many doubters spread fearful rumors about the edge / At last / From nowhere / Welcome winged creatures appeared / Signifying momentous success.

People’s ability to comprehend and remember this passage is much greater when they are told that it is about Christopher Columbus discovering America than when they are not given any contextual hints as to its meaning. Furthermore, when readers are not given any contextual cues for the interpretation of the passage, they typically attempt to subjectively conjure up a viable context from their own idiosyncratic knowledge of the world—a context that might enable them personally to derive meaning from the passage. To the extent that they are able to develop such a context, they are able to comprehend and remember the paragraph.

These examples serve as elementary illustrations that when we process and interpret events, we do so from our perspective of the world. It is, then, through a synthesis of our cognitive knowledge of the world with present input that we derive meaning from our experiences. Furthermore, when we do not have the appropriate context for an event at our cognitive disposal, we often adopt an alternative contextual interpretation whereby meaning might be derived.

* * *

Many years ago, cultural pundits began to intimate that Americans increasingly would find themselves living in a psychologized society. This point was stated poignantly by C.G. Ellison in an essay in Current Perspectives in the Psychology of Religion (1977): “Psychology has grown into a giant during the twentieth century. No other age has witnessed such intense concentration upon the nature and functioning of ‘homo sapiens.’ Psychological terminology has become an integral part of the common vernacular and psychological concepts strongly influence contemporary thought.”

In the midst of this “psychologization” of Western language and thought, we have more and more come to view the vicissitudes of life in terms of psychological categories. Or, as argued by the authors of the unlikely best seller Habits of the Heart: Individualism and Commitment in American Life (1985), it is the therapist (along with the manager) who largely defines what constitutes effective living in American culture. The authors of this book offered the following statement as a succinct summary of this therapeutic mind-set: “Like the manager, the therapist is a specialist in mobilizing resources for effective action, only here the resources are largely internal to the individual and the measure of effectiveness is the elusive criterion of personal satisfaction. . . . Indeed, the very term therapeutic suggests a life focused on the need for a cure. But a cure of what?” (italics mine).

We find within this definition the following components of the therapeutic mind-set. First, as it attempts to provide a viable framework for the interpretation of reality, this set of cognitive knowledge structures concentrates on the internal psychological and emotional workings of the individual. Second, this mind-set emphasizes the need for men and women to be cured / healed. Third, the therapeutic schemas suggest to the twenty-first century interpreters of events (that is, to us) that the end result of this healing process should be fewer blocks to personal growth, greater personal satisfaction, less personal suffering, and a greater sense of personal well-being. Fourth, this therapeutic way of perceiving reality emphasizes a utilitarian view of life, a view in which virtually all human endeavors (from reading Scripture, to worship, to personal relationships) are evaluated based on criteria of psychological effectiveness.

As we move forward in the twenty-first century, there is little to suggest that this psychologization of Western thought and language has abated. In Modernizing the Mind: Psychological Knowledge and the Remaking of Society (2002), author Steven C. Ward put it this way: “Throughout this book I have tried to illustrate how over the course of the last century psychological categories and practices became ‘naturalized.’ As this happened, psychology, like other naturalized ideas and categories, ‘disappeared into infrastructure, into habit, into the taken for granted.’ Psychology’s presence in schools, workplaces, and homes is now an ordinary and seemingly indispensable feature of the cultural landscape. . . . Today, psychological knowledge and categories seem ‘intuitively right and reasonable because they are rooted in everyday life.’”

* * *

The encroachment of a psychological mind-set into the day-to-day workings of Western culture is obvious. It can be argued, in fact, that therapeutic schemas have grown to such a stature of prevalence, consequence, and acceptability within the culture that they dwarf all other views of reality. As a result, it should not be at all surprising to find a growing presence of therapeutic schemas within the Christian churches. The authors of a 2005 book titled Soul Searching: The Religious and Spiritual Lives of American Teenagers have suggested that this may be the case much more than had previously been suspected.

Soul Searching is based on in-depth interviews with more than 265 teens from various faith traditions. The authors of the book begin by offering some encouraging findings about young people and their faith. As these teens responded to specific questions in the interview sessions, for example, the vast majority identified themselves as Christian. Furthermore, many of the teens reported being involved in such religious activities as regular church services, youth retreats, youth rallies, and service projects; they saw participation in an organized faith community as an important way to have ongoing contact with influential adults; and most of them responded with positive attitudes toward their religion and their parish or congregation.

In an effort to further gauge the religious lives of these young people, the authors proceeded not only to record the teens’ overt responses to the specific interview questions, but also to take note of the words, terms, and phrases that most often emerged in the transcripts of the interview sessions. These analyses begin to reveal quite a different picture of the spiritual lives of young people in the United States.

For the most part, the language of the young people did not reveal a faith that revolved around traditionally relevant Christian themes. The teens’ expressed belief systems did not center on being faithful to God, or repenting for sin, or building character through steadfastness. Instead, for most of them, what was essential to their faith was “feeling good, happy, secure, at peace”—what the authors term the “therapeutic benefits” of religious involvement.

More specifically, the authors report that the phrase “the grace of God” was mentioned by these 265 teens a scant three times. The same was true for “honoring God with your life” and “the importance of loving your neighbor.” Similarly, “the justice of God” was mentioned only twice, and several historically significant Christian themes—self-discipline, sanctification, social justice—were not mentioned at all. In contrast, 112 teens mentioned personally feeling happy or being made happy through faith, and 99 different teens discussed feeling good about one’s life. In fact, the specific phrase “feel happy” was used more than 2,000 times during the interviews.

In a particularly pithy summary of these interviews, the authors offer the following: “When teenagers talked in their interviews about grace, they were usually talking about the television show Will and Grace, not about God’s grace. When teenagers discussed honor, they were almost always talking about taking honors courses or making the honor role at school, very rarely about honoring God with their lives. When teens mentioned being justified, they almost always meant having a reason for doing something behaviorally questionable, not having their relationship with God made right.”

One might be tempted to speculate that although such religious trends may typify the spiritual lives of many (if not most) young people, they certainly are not an accurate reflection of the more seasoned faith of adult believers in the United States. The authors of Soul Searching beg to differ, however. They are quick to assert that the type of belief system encountered among the teens who were interviewed is not restricted to the young; rather, it “is also a widespread, popular faith among very many U.S. adults. Our religiously conventional adolescents seem to be merely absorbing and reflecting religiously what the adult world is routinely modeling for and inculcating in its youth.”

* * *

One need not be a seer to perceive the infiltration of this therapeutic mentality into sectors of the twenty-first century Church. One need only be cognizant, first, of the growing number of committed Christian men and women for whom the virtues of courage, fortitude, compassion, and charity have become blurred in the midst of their psychological and emotional misgivings; second, of the number for whom the pursuit of goodness, truth, beauty, and moral character has been supplanted by a search for personal satisfaction; third, of those for whom thoughts of loyalty, duty, and commitment have been recast in terms of personal growth and well-being; and, fourth, of those for whom suffering has become merely an indication that something is personally not right and needs to be cured or fixed. One need only listen to some of the messages emanating from the pulpits in today’s churches—messages in which personal hurts may be portrayed as greater pitfalls to the Christian walk than personal sins; where psychological wholeness is emphasized more than sanctity; where the presence of authority may be viewed as spiritually more destructive than the presence of Satan; and where believers may be encouraged to find themselves more than they are encouraged to find God.

These comments are not meant to suggest that the gospel is opposed to personal well-being and the diminution of suffering, only that such therapeutic categories may not provide legitimate criteria for evaluating the efficacy of one’s decisions, behaviors, and relationships. Nor are these comments meant to suggest that solid psychological intervention may not be a vital part of the spiritual growth process for some Christian men and women, only that the standards against which the success of such interventions are evaluated may not be therapeutic ones. In other words, the gospel has never claimed, in our classical, Judeo-Christian tradition, to provide another path to “the therapeutic good life.” Christianity has not claimed uniqueness for its therapeutic benefits.

* * *

Years ago, when I was a graduate student in Chicago, there was an article in one of the local papers about a wealthy woman who had instructed her chauffeur to drive her to a new McDonald’s that had opened in a downtown shopping mall. On entering the restaurant, the woman took a seat at an open table and waited. No one came. The longer she waited, the more upset she became. Finally, she asked to see the manager to complain about the poor service. As this woman attempted to understand her experience at McDonald’s, she cognitively accessed her understanding of how restaurants work. Because the restaurant schema she had developed over the years was incorrect for a visit to McDonald’s, however, she misinterpreted the situation and, accordingly, organized her actions mistakenly.

There are many men and women in the Church today who, like the woman at McDonald’s, cognitively access their available schemas in an attempt to make sense of their life circumstances but come up short because the therapeutic schemas with which they have been equipped fail to adequately represent the reality of our human condition. One should not be surprised to find that when this happens, traditional Christian ideals such as selflessness and self-denial, duty and fortitude, repentance and reconciliation, and sacrifice and sanctification are scarce commodities, even among some of the most committed of Christian men and women.

At present we face a challenge of great importance. We face it in our parishes and congregations, in the everyday lives of our fellow Christians, and, as a consequence, in our society. Increasingly, the ideological and cultural character of churches today is being formed by a worldview that espouses not virtue, but psychological satisfaction and well-being. It is a worldview in which the moral excellence that can be found in suffering often is negated in the hurried search for relief; it is also a worldview in which the greatness of character that derives from habitually meeting all of one’s duties with excellence is challenged by the voice of therapeutic reason: “This is hard. Besides, what am I going to get out of it?” Is it any wonder that people decide to shift their church affiliation based on such criteria as enjoyable music, lively homilies, or a satisfying Sunday-morning coffee-and-donut experience? Or that people who find themselves in personally unfulfilling marriages conclude that God could not possibly be in such a marriage?

Both Christianity and modern psychology provide ideologies for the understanding and guidance of the interior life, and many of us find ourselves in singular positions as we strive to bridge these sometimes disparate disciplines. The English geographer Sir Halford John Mackinder stated in 1887 that “Knowledge is one. Its division into subjects is a concession to human weakness.” Conceding to our human weakness, we must explore carefully the differences between these two models of care for the interior life. The problems of human existence take on a distinctively different significance when viewed through a psychological mind-set intent on the attainment of positive internal states than when viewed through the eyes of one who realizes the inherent suffering in the ongoing personal repentance, conversion, and regeneration asked of us by Christ. Conceding to our human weakness, we should be the first to pursue all that psychology has to offer in the understanding of the human condition; but, at the same time, we should be the last to blur the distinctiveness of the Christian gospel.

John Buri is a professor of psychology at the University of St. Thomas in Saint Paul, Minnesota. He is the author of over fifty professional articles and one book (How to Love Your Wife). He also teaches a course in the Catholic Studies Program at St. Thomas in which traditional Thomisic moral theology is compared with modern psychology.

Q. ‘I’m creating a custom activity designer in VS and I want to add an icon to it. How do I do that?’

A. The class ActivityDesigner (System.Activities.Presentation) has a property called Icon.

    public DrawingBrush Icon { get; set; }

You can set it from XAML

<sap:ActivityDesigner x:Class="WorkflowConsoleApplication6.ActivityDesigner1"

    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"

    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"

    xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation">

    <sap:ActivityDesigner.Icon>

        <DrawingBrush>

            <DrawingBrush.Drawing>

                <GeometryDrawing Brush="Coral">

                    <GeometryDrawing.Geometry>

                        <GeometryGroup>

                            <EllipseGeometry RadiusX="5" RadiusY="5"/>

                        </GeometryGroup>

                    </GeometryDrawing.Geometry>

                </GeometryDrawing>

            </DrawingBrush.Drawing>

        </DrawingBrush>

    </sap:ActivityDesigner.Icon>

 

or you can set it in your constructor (here loading a resource from a resource dictionary in the same assembly)….

 

        public ActivityDesigner1()

        {

            InitializeComponent();

            Uri uri = new Uri(

                "WorkflowConsoleApplication6;component/Dictionary1.xaml",

                System.UriKind.Relative);

            ResourceDictionary rd =

                (ResourceDictionary)Application.LoadComponent(uri);

            this.Icon = rd["CoralIcon"] as DrawingBrush;

        }

 

but the result is much the same:

 

Q. ‘How do I show a custom activity with no icon or DisplayName at all (just whatever UI I provide)?’

A. To build a designer starting with no visuals at all, you can subclass from WorkflowViewElement instead of ActivityDesigner.

Q. ‘Is there a way to override the default activity icon at a global level?’

For the Out-of-box activity designers? No. For your own custom activity designers, as above.

Q. ‘How do I set a custom toolbox icons in a Workflow Designer rehosting application?’

(The assumption in this answer is that you are using the Toolbox, ToolboxCategory, and ToolboxItemWrapper classes in System.Activities.Presentation, and not some custom solution.)

You can set the BitmapPath property on a ToolboxItemWrapper.

Q. ‘This BitmapPath setting experience is extremely limited, it blocks me from using vector graphics, and it forces me to create a lot of icon files. Can’t I do something better?’

For this answer it helps to notice that ToolboxCategory and ToolboxItemWrapper represent the items in your toolbox, however they are not actually Visuals, and they are not in the WPF visual tree. Instead their role is to act as the toolbox category/item ViewModel (as in MVVM), and supply property changed notifications with WPF uses when binding.

The Toolbox class on the other hand is part of the visual tree, and it also exposes some other customization points via properties that you can use to drastically modify the toolbox item appearance or templating behavior: CategoryItemStyle, CategoryTemplate, ToolboxItemStyle, and ToolTemplate. The default template for rehosted applications just consumes the bitmap provided on the ToolboxItemWrapper like this.

    <Image x:Uid="Image_1" Grid.Column="0" Source="{Binding Path=Bitmap, Converter={StaticResource ImageConverter}}"/>

With this knowledge, you can replace the default template with a new template which binds an image based on some other property of the ToolboxItemWrapper such as it’s name:

    <Image Grid.Column=”0”, Source={Binding Path=ToolName, Converter={StaticResource CustomToolImageConverter}}”/>

Of course since now you are defining the template you don’t have to use <Image> - you can even use something totally different. Here’s an example XAML using DrawingBrush…

<sapt:ToolboxControl Grid.Column="1" x:Name="toolbox">

  <sapt:ToolboxControl.Style>

    <Style TargetType="{x:Type sapt:ToolboxControl}">

      <Style.Resources>

        <local:ToolboxContentConverter x:Key="CustomConverter"/>

        <DataTemplate x:Key="IconTemplate">

          <Rectangle Width="20" Height="20" Fill="{Binding}"/>

        </DataTemplate>

      </Style.Resources>

      <Setter Property="ToolTemplate">

        <Setter.Value>

          <DataTemplate DataType="{x:Type sapt:ToolboxItemWrapper}">

            <Grid>

              <Grid.ColumnDefinitions>

                <ColumnDefinition Width="Auto"/>

                <ColumnDefinition Width="*" SharedSizeGroup="toolLabel"/>

              </Grid.ColumnDefinitions>

              <ContentControl Grid.Column="0"

                Content="{Binding Path=ToolName, Converter={StaticResource CustomConverter}}"

                ContentTemplate="{StaticResource IconTemplate}"/>

              <TextBlock Grid.Column="1" Text="{Binding Path=DisplayName}" Margin="5,1,5,0"/>

            </Grid>

          </DataTemplate>

        </Setter.Value>

      </Setter>

    </Style>

  </sapt:ToolboxControl.Style>

</sapt:ToolboxControl>

 

And in the toolbox, it comes out looking like this (the icon size is modified from standard in the DataTemplate used for applying the CustomConverter (which is returning a DrawingBrush):

 

Q. ‘I want to show all the Out-of-box activities with their out-of-box icons as seen in VS. But in my rehosting application, not VS.’

I worry this question could tread on shaky legal ground. I haven’t checked the facts yet, but if I remember correctly the Icons are in the Microsoft.VisualStudio.Activities assembly, and Microsoft.VisualStudio.Activities.dll is not one of the DLLs licensed for redistribution. OK, just wanted to add that warning before I post the link (again) to the howto. Here it is: Displaying .NET Framework 4 Built-In Workflow Activity Icons in a Rehosted Designer.

Enjoy!

BayTech Webs, www.baytechwebs.com, is ranked as Top web development firm and a Gold Award Winner of Horizon Interactive Award in 2008 & 2009. The company specializes in custom web solutions to fulfill clients' web based application across all channels. The company has a fast growing team of world-class professionals specializing in consulting, design, development and web-marketing. The company delivers unprecedented web 2.0 solutions across a wide range of industries.

Responsibilities:
1. Responsible in the entire sales / web development process including technical coding and architectural guidance, serve as primary Account Rep to prvide scoping towards website development / or design requirements. Collaborate with Designers to meet with clients in scoping design concepts, visual graphics, business logic, navigation maps, style guidelines, and other UI design documents.

2. Interact with clients, responsible for understanding client’s business logic and scoping client's needs, participate in overall architecture design; contribute to DB schema and architecture back office design.

3. Brainstorming and defining WUI features and provide guidance/ assistance to the production team

4. Reponsible to draft complicated back-end wire frames and User Documentation(s)

5. Provide a comprehensive website development project analysis toward overall project timeline and costs (Project Scoping)

6. Responsible in drafting / presenting project proposal by analyzing each different unique back-end functionality subject to RFP/RFI request

7. Provide both pre-sales & project management consulting and collaborate with production team, senior project managers and development teams through the development timeline to ensure project requirements are clearly defined, met and understood.

8. Coordinate with visual designer / senior programmers to ensure that both wireframes & mockups meet the objectives of the user experience; satisfy with the overall programming logic and backend UI consistence. Monitor project schedules and timelines to identify and meet critical milestones and deliverables for multiple website development projects with competing priorities and deadlines. Provide support / monitoring to ensure the (Deliverables) are finalized, sign-off and launch to the public domain. In addition to the above, must also provide coding efforts to all assigned data-base driven website projects.


Requirements:
1. BS major in Computer Science with the following prefer courses completed:
(Database Design, Software Project, Data on Web, Multimedia System, Process and Quality Management, Software Requirements, Software Verification, Validation and Testing.)

2. Minimum 3-5 years of work experience in the software development / web application development with proven logical and analytical problem solving skills. 3 years of Industrial hands-on professional experience with Application Development

3. Rich experience of Clients Business Logic for various industries.

4. Deep knowledge of the following required area:
a. Language: PHP, Asp.Net, HTML, XML, AJAX/ JavaScript
b. Databases: Microsoft SQL Server, MySQL.

5. Collaborative team player with excellent communication skills; ability to multitask, and thrive in a fast-paced environment, while meeting deadlines and ability to function under pressure in a highly dynamic environment

6. Must be able to work independently with minimum supervision

7. Enthusiastic Highly motivated with strong negotiation skill, Capable in managing the entire project flow.

8. Must have high ethics, integrity and interpersonal skills and straight talk without politics

9. W2 or current paystubs is required

Firm is seeking at least two motivated self-starters who will be part of a team of highly skilled, professional and experienced architects, interior designers and planners. The firm offers a diversified experience of multiple project types in various stages of design and development. We specialize in planning, architecture, environmental graphics and development consulting on TOD, mixed-use, multifamily, commercial and urban infill. Commercial space planning and 3D/2D rendering are a plus.

Desired candidate may be involved in schematic design to the preparation of construction documents, assist in the preparation of specifications, and provide hands-on construction administration. Candidate must have an understanding of planning, programming and design principles, knowledge of building and zoning codes, experience with architectural budgeting, and the ability to design within project schedules and budgets. Canidate must be able to produce all design documents types. No management only positions, all canidiate must be able to perform.

Additional Requirements --
* Must have 4 to 10 years relevant experience in an architectural/interior/planning or similarly oriented practice.
* Possess a Bachelors or Masters of Architecture.
* Must have or be able to achieve Architectural or AICP Registration.
* Required proficiency in AutoCAD and or Revit Architecture
* Experience with Building Information Modeling (“BIM”)
* Working knowledge of all CAD / BIM related platforms including but not limited to AutoCAD, Revit, 3D Studio Max, Photoshop, SharePoint, Sketch-up, Accurender, Microsoft Office and Outlook
* Strong communication, people management and time management skills required
* U.S. Citizenship Required

Benefits include but are not limited to:
* Competitive Salary
* Excellent Healthcare Benefits (medical, dental, vision, prescription)
* 401(k) Retirement Program
* Authorized Time Off including Paid Holidays, Vacation and Personal Leave
* Professional Development Program

If you meet the above and have a interest please inquire by writing a letter with target salery and resume.

Regal Wine Company is California’s preeminent luxury wine distributor concentrating entirely on premium brands of outstanding quality from around the world. We are committed to providing world-class customer service to our valued customers across all relevant market segments. Acting as entrepreneurs our company encourages individual initiative and is determined to grow our creative resources.

POSITION SUMMARY:
Manage assigned territory of Chain Accounts. Sell displays, maintain 100% distribution, and complete administrative responsibilities for the territory.

ESSENTIAL DUTIES/RESPONSIBILITIES:
• Maintain 100% shelf distribution of all authorized items for each chain.
• Set shelves according to store schematics.
• Cut-in new item distribution and DSD items, maintain current pricing information.
• Rotate vintages, fill shelves, fill cold box, build displays, and maintain Royal Wine Company housekeeping standards including dusting and cleaning tables.
• Establish & maintain rapport with chain store managers, liquor department managers, and other store personnel.
• Sell current programs for mass floor displays, build displays, and put up P.O.S. and prices.
• Keep track of individual Travel & Entertainment expense reports and Samples, staying within budgets.
• Responsible for submitting Travel & Entertainment expense reports by the end of the month.
• Complete daily preplans, recaps, and distribution reports.
• Follow the 10 steps to a successful sales call.
• Attend and contribute to weekly team meetings, participate in role-play presentations, review displays and case goals vs. actual.
• Must carry point of sale merchandise in motor vehicle at all times.
• Responsible for carrying and cell phone at all times in order to communicate with accounts and management at any time during working hours.
• Other duties as assigned by Supervisor.

JOB REQUIREMENTS:
Previous sales experience preferred, but not required. Successful candidate must: be at least 21 years of age; possess a current driver’s license; maintain current auto insurance coverage; be able to repetitively lift a minimum of 50 lbs; be able to bend, squat, climb, kneel, twist and lift repetitively; possess excellent verbal communication, written communication, and listening skills; have the ability to work in a fast-paced environment and be a team player who can work well with minimal supervision; and demonstrate the ability to multi-task and respond quickly to demanding situations. A qualified candidate will take a creative approach to sales. This position requires the availability to work flexible hours and weekends, when necessary, for special events. Candidate must be willing to reside within territory.

If you would like to apply for this position, please submit your resume online at:

https://www.abso.com/jobboard/?JOBBOARDID=549&JobDetail;=69720

Schnitzer Steel Industries, Inc. is one of the largest steel recycling companies in the United States. We are experiencing aggressive growth and are seeking a dynamic individual to join our Metals Recycling Business team. This position is an exciting opportunity for the right person looking for stable, long-term growth potential. Schnitzer Steel Industries places a high value on its employees and their career development. Please visit our website at www.schnitzersteel.com for detailed information regarding our company profile, more information about this position, and to complete an online application.

General Position Summary:
General mechanic work with some welding capabilities.

Internal Control Responsibilities:
Communicate upward problems in operations, noncompliance with the code of conduct, or other policy violations or illegal actions.

Job Scope:
Troubleshoot and repair equipment and machinery such as, but not limited to excavators, material handlers, bucket loaders, bull dozers, haul trucks, forklifts, boom lift and stationary machinery. Repair/replace components on heavy equipment and machinery. Rebuild components for equipment. Perform preventive maintenance

This position operates within somewhat broad, but established parameters. Employee will be trained on various pieces of equipment and areas of the scrap yard. Must be able to adapt to new functions and environments when needed.

Essential Functions:
- Able to stand for up to 12 hours.
- Be able to bend at waist.
- Good eye hand coordination.
- Knowledge of basic mechanics and computer applications
- Must be able to work with both hand and power driven tools.
- Test equipment and mobile machinery after repair
- Provide maintenance trouble shooting for hydraulic systems on mobile equipment
- Read and comprehend maintenance schematics

Marginal Functions:
- Provide backup at other locations of the yard besides what is originally assigned.
- Perform special projects as assigned.

Qualifications:
- Experience in scrap yard preferred but not necessary. Knowledge or and experience of heavy equipment and millwright duties. Technical training and/or degree in Diesel Tech preferred but can be substituted for on the job experience. Basic knowledge of computer applications.
- Ability to carry out instructions provided in written, oral or diagrammatic form. Ability to speak and understand English at a basic level.
- Valid drivers license
- Experience with maintaining and troubleshooting and repair of heavy equipment including, but not limited to cranes, excavators, loaders and forklifts.
- Be able to work above ground level on scaffolding, aerial equipment and other various equipment.
- Be able to complete safety inspection on equipment.

Physical Activities Required to Perform Essential Functions:
Job requires ability to lift up to 100 pounds. Also as a part of regular work routine, be able to stand and walk, with frequent lifting, guiding and/or carrying of light-weight materials or equipment. Ability to: twist upper body frequently, bend at waist, kneel, crouch and negotiate uneven surfaces; use hands repetitively for coordinated finger movements; communicate by speech and hearing in person at a basic English proficiency.

Supervisory responsibility:
This position has no supervisory responsibilities.

Interpersonal Contacts:
Internal contacts include the other laborers and Yard Supervisors. External contacts include customers bringing in scrap to the yard.

Job Conditions:
Work may be performed in cramped or awkward positions. Occasional exposure to safety hazards, noise and temperature extremes. Work pressure, disturbances of work flow, and/or irregularities in work schedule are expected and occur on an intermittent basis.


PLEASE NOTE: The above statements are intended to describe the general nature and level of work being performed by people assigned to this job. They are not intended to be an exhaustive list of all responsibilities, duties and skills required.

If you meet the qualifications listed above, please apply as follows:

Go to www.schnitzersteel.com to apply online directly to this position and to complete an on-line application.
OR

Go to www.schnitzersteel.com to download an application and mail it along with a current resume to:
Schnitzer Steel Industries, Inc. Attn: Human Resources - Portland Yard, PO Box 10047 Portland, OR 97296.

Applications must be filled out in their entirety for applicants to be considered for this position.

Are you floored by the recent Supreme Court decision unleashing corporate contributions to political campaigns?

Have you always suspected that voting in Congress is influenced by the contributions of interest groups, but never had the data to prove it?

Does a chance to work magic with that data to develop sophisticated tools to support high-profile research fill you with glee?

MAPLight.org, a cutting-edge non-partisan research organization located in Berkeley California, is looking for the right stuff in a PHP/MySQL developer. We need someone who speaks in SQL and dreams in PHP. Someone who can look at gigabytes of data and quickly develop innovative strategies for getting the most out of our research and analysis tools.

Responsibilities:
* Work in a small, dynamic team to develop new web-based tools to support a talented research team and an audience of millions.
* Help to develop and optimize database schema and queries.
* Develop and improve processes to glean data from a wide range of data sources that is the backbone of our research.
* Enhance our Web 2.0 applications with top-notch PHP coding and client-side scripting.

Qualifications:
* 5+ years web development, with a strong focus on LAMP and open source tools.
* 5+ years database experience in MySQL or Postgres.
* Very strong SQL skills.
* Expertise with PHP, DHTML/CSS/Javascript, XML.
* Knowledge of Drupal development.

A knowledge of the American political system (or a desire to learn) is preferred, as is a passion for openness and transparency.


This is a full-time position with salary and benefits commensurate with experience -- we are not interested in a contract situation at this time. MAPLight.org is located in downtown Berkeley, California (close to BART). Telecommuting is an option for exceptional candidates, but be prepared to put some effort into proving that are the right person for that option.

Interested candidates should submit a cover letter and resume to eric [at] maplight [dot] org.

PLEASE INCLUDE A CODE SAMPLE AND URLS TO YOUR ONLINE WORK


About MAPLight.org MAPLight.org, a groundbreaking public database, illuminates the connection between campaign donations and legislative votes in unprecedented ways. Elected officials collect large sums of money to run their campaigns, and they often pay back campaign contributors with special access and favorable laws. This common practice is contrary to the public interest, yet legal. MAPLight.org makes money/vote connections transparent, to help citizens hold their legislators accountable.

MAPLight.org’s small, effective team has won numerous awards, including a 2008 Webby award nomination for best Politics website. Learn more at http://www.maplight.org.

Database and Interactive Tools Development

The Position
- SQL Server Database Development and Architecture
- ETL Developer
- Tools and interface developer
- This position reports to the VP Engineering and is responsible for defining, specifying, and building databases, ETL tools, and interfaces that form the foundation of the company’s service offerings.
- The position is contract-to-permanent, and could be full time immediately for the right person.
- This person is expected to be an individual contributor by both participating in the design of the database strategy and by executing on those plans.
- This highly valued role will participate in management meetings for product development and be expected to provide management with executive level insight, planning, and execution

Current System
- The company has an immediate need for a superior database developer to extend and expand its v.1 database system.
- The databases form the underpinnings of the company's commercial products and supports commercial users and internal resources.
- Data streams in and out for real time production calculations
- Ability to slice and dice for performance analytics.
- The database requires the ability to accommodate terabytes of data as the company's models are further developed.
- Currently MYSQL, flexibility to change direction as size and speed requirements change.
- Minimal access and analysis tools are currently a part of the system, many planned.

Required Skills
- SQL, SQL Server, ETL
- Performance Tuning
- Design/Maintain Indexes
- Verifiable experience developing, administering, and maintaining databases in a high volume production environment
- Track record for implementing optimal policies and procedures for ensuring the security and integrity of the company databases.
- Experience improving database performance, capacity, and reporting systems.
- Ability to interact with clients to ensure the current data systems as well as future development of new systems that meet their business objectives and needs.
- Database Design, VLDB (Very Large Database Development, SQL, SQL Server, ETL, Performance Tuning, Design/Maintain Indexes, Database Design, VLDB (Very Large Database)

Desired Skills
- Working knowledge of atmospheric data strongly desired.
- Star Schema Design
- Reporting tools and Java based tools development

The Company
- The world leading team of scientists and high performance computing engineers has been developing atmospheric models for almost two decades.
- Our models have been mathematically verified as the most progressive and most accurate in the world for providing short term weather forecasts.
- The company has been providing commercial services to the power sector for over two years.
- The company seeks to expand its team and its product line.
- We are seeking the best people in their respective expertise.
- The company respects creativity, curiosity, and a can-do attitude. A lack in these areas, and expertise alone will not land this job.

Location and Commitment
- The company has offices in Santa Cruz CA, Las Cruces NM, and Fort Collins CO.
- This role is a contract to hire position
- Could be FTE immediately for the right person
- Work from home office as appropriate is OK
- Management experience desired.
The company prefers to hire senior managers and contributors, then to fill out the team below them, rather than building teams and seeking managers. This is an opportunity to participate on the ground floor with a company that has already proven itself with investors, where the hard science is done, and where the delivery, management, and client side tools need to reflect the superior nature of the processes beneath them.

We expect a large response to this post. While we will be proactive in responding to CV’s and resumes sent prior to March 16, 2010, we appreciate your patience in hearing back from us.

JOB DESCRIPTION: Senior Engineer - Online Games
General Information
Job Title: Lead Engineer - Online Games Department: Engineering
Location: S.F. Bay Area Travel Required: Minimal
Position Type: Full Time Employee Salary Range: Depends On Experience
Reporting Relationship: CTO N/A
Send Applications To...
Name: Barry Holroyd
Email: barry.holroyd@mashermedia.com

U.S. Post: 405 Hedgerow Court, Mountain View, CA 94041-1628
Company Overview
Masher Media is an early stage startup which is in the process of producing a revolutionary new MMOG for kids called MyMiniPeeps. MyMiniPeeps will define the next evolution in kids’ websites -- it is a unique, social and safe online virtual world and community for kids ages 6-13 that will raise the bar for innovative concept, level of interaction, and creative and aesthetic content on the web. MyMiniPeeps will be one of the first fully immersive 3-D multiplayer environments for this demographic, playable on the browser, through a download to the hard-drive and on the iPhone.
Job Description
Summary
Masher Media has produced a solid Alpha release of MyMiniPeeps and is now ready to staff the core engineering team that will architect and build out the version 1.0 platform.

The person filling this position will report directly to the CTO and will be the first member of the engineering team. He/she is expected to have significant impact on the architecture and development of the product. The ideal candidate will have significant experience developing MMOGs and will be comfortable handling a broad range of hands-on activities -- both client-side and server side development will be required, as well as the development, integration and configuration of various ancillary services such as registration, buddies lists and chat filtering.

This is a key position for the company -- we are looking for someone who is captivated by online games and virtual worlds in general and who is excited by joining a company which is expected to have a real impact on the industry.
Primary Duties
This person will be the senior hands-on technical member of the team and will be responsible for understanding all aspects of the system, including the client front end as well as the server back end (real time server, web services and database). Specific responsibilities include:
• Implementation of 3D MMOG features
o Server-side implementation of game infrastructure (e.g., shard selection), game logic, persistence, status data collection, etc.
o Client-side virtual world development and coding
• Implementation of ancillary functions (either via the integration and configuration of third party packages or developed internally), e.g.: Parent Portal, Social Portal, Micro-transactions
• Assisting third party developers with all aspects of development and integration, as necessary.

Secondary Duties
This person may also be responsible for some of the the following activities, (the exact set is TBD).
• iPhone application
o Modifications to implementation to support iPhone version of game.
o Writing a client-side library for communication with the server-side platform.
• Extension of the online game web site, MyMiniPeeps.com.
Qualifications
Skills - Required
The applicant should have significant experience in most, if not all, of these areas.
• 3D MMOG development
• Unity 3D world editor
• Microsoft development technologies (C#, Visual Studio, SQL Server, MS Server 2008, ASP.NET, etc.)
• Design and programming of multi-tier, multithreaded applications, e.g.:
o Network communications (e.g., socket servers)
o Database schema design
o Transaction processing (in-game, credit card, third-party systems)
o Threads and locking
Skills - Desired
The following skills and experience would be very helpful for this position.
• Game analytics tracking and reporting
• System monitoring and performance
• Enjoyment of playing online games
• Strong communication skills; team-oriented
• Patch management and deployment
• Cloud computing
• Software: Crisp text filter
• Designing and/or programming games for children
• Languages: PHP

Souvenir shops in South Africa are full of lamps made out of ostrich eggs. The eggs are so big and strong that you can carve and cut intricate designs into their shells. The egg's contents are emptied through a hole and a bulb can be inserted instead, casting pretty shadows on walls and ceilings. The results are a big draw for modern tourists, but ostrich eggs have a long history of being used as art in South Africa. The latest finds show that people were carvings symbolic patterns into these eggs as early as 60,000 years ago.

Pierre-Jean Texier from the University of Bordeaux discovered a set of 270 eggshell fragments from Howieson Poort Shelter, a South African cave that has been a rich source of archaeological finds. Judging by their patterns, the fragments must have come from at least 25 separate eggs, although probably many more.

Texier says that the sheer number is "exceptional in prehistory". Their unprecedented diversity and etched patterns provide some of the best evidence yet for a prehistoric artistic tradition. While previous digs have thrown up piecemeal examples of symbolic art, Texier's finds allow him to compare patterns across individual pieces, to get a feel of the entire movement, rather than the work of an individual.

As you might expect, the millennia haven't been too kind to the shells but even so, their etchings are still well preserved and Texier even managed to fit some of the pieces together. Despite the variety of fragments, their patterns fall into a very limited set of motifs produced in the same way - a hatched band like a railway track, parallel(ish) lines, intersecting lines, and cross-hatching. It's possible that, once assembled, these elements would have combined into a more complex artistic whole but Texier notes that he has never found a piece with more than one motif on it.

Much like any other artistic movement, the shell designs had rules that everyone abided by, but also room for some stylistic latitude. There were four main styles but each had some wiggle room in their execution. The railway track, for example, had lines of different lengths and intersection angles. While some fragments had just one track, others had two or three.

These traditions also seem to have risen into fashion and fallen out of them again. The railway track motif only turns up in the lower parts of the cave and not in eggshells from the upper reaches. Go further up, and it's the parallel lines motif that dominates. To Texier, the shell motifs are evidence that these prehistoric humans were more than capable of symbolic thought, of using patterns to transform ordinary items into specific and unique ones.

Today, hunter-gatherers in the Kalahari still collect ostrich eggs, for food, as beads, or as water containers. They puncture a small hole in the top of the egg, empty their contents and fill them with water. Texier says, "Some Bushmen groups, like the !Kung, use a graphic and schematic tradition to communicate collective identities as well as individual ones." He thinks that the prehistoric hunter-gatherers may have used the eggs for a similar purpose. Their engravings may have been used to intentionally mark ostrich eggshell containers and some of the fragments show signs of forced entry, typical of modern egg-based water containers. To Texier, they were "elements of a collective and complex social life", signs of a modern human intelligence operating tens of thousands of years ago.

Christopher Henshilwood, another anthropologist working in South Africa, was also impressed. "Based on sheer numbers the evidence for deliberate decoration and symbol use is compelling." He adds, "Perhaps the greatest challenge still is in explaining why the tradition of engraving appears to... appear, disappear and then reappear at different times and places."

Several years ago, Henshilwood found a different piece of abstract art at another South Africa cave called Blombos. It was a slab of ochre, also covered in geometric carvings and dating back to 70,000 years ago. "Is there really a connection betweeen the engraving traditions at 100 ka, 75 ka, 60 ka and those in the Later Stone Age?" he asks. "And why are engraved ostrich eggshells and ochre only found at some Middle Stone Age sites and not others?" 

Reference: Texier et al. 2010. A Howiesons Poort tradition of engraving ostrich eggshell containers dated to 60,000 years ago at Diepkloof Rock Shelter, South Africa. PNAS. http://dx.doi.org/10.1073/pnas.0913047107 If this link isn't working, read why here

More on art and archaeology: Prehistoric carving is oldest known figurative art

   

Company Overview

Digital Aisle LLC is an industry-leading media company paving the way in next-generation digital shopper-marketing. The world’s largest Fortune retail companies rely on Digital Aisle because our proprietary solutions are revolutionizing the way to communicate with consumers at the critical point-of-purchase. The Digital Aisle platform has proven to deliver significant results, including higher consumer satisfaction and increased sales. To learn more, visit www.digitalaisle.com


Job Description

Digital Aisle is seeking a local Chicago area candidate to fill a Senior .Net Application Developer position. This is a full time position based out of Woodridge, IL with minimal domestic travel. The desired candidate must be highly motivated and seeking a long-term career opportunity. The candidate must have experience designing and developing Desktop and Web applications using various Microsoft technologies with emphasis on rapid application development. The candidate must be able to work efficiently in a small team environment, be willing to learn new technologies. The position will have extensive participation in the end-to-end software development process and regular client interaction.

Job Experience - Specific Skills

• At least 6-8 Years of Experience in using Microsoft .NET (versions 2.0 and 3.5) - C# and/or VB.NET to build Windows Forms based Desktop as well as ASP.NET based Web applications.
• Experience in the use of ADO.NET and ORM APIs such as NHibernate for Database Access
• Experience in developing Windows Native Services
• Experience in developing Web Services using the Windows Communication Framework
• Experience in use of Microsoft SQL Server (2005 and 2008), Oracle (9i or 10g), and MySQL – Entity Modeling, Transact SQL, Views, Stored procedures, Jobs, Triggers, etc.
• Experience in developing ActiveX Controls using either .NET or MFC/ATL
• XML Schema Design/XML Parsing/XSLT Development/XQuery/XPath

Job Experience – General Skills

• Proficient rapid application developer who can get going immediately
• Proficient in Designing and Implementing n-Tiered applications
• Proficient in Object Oriented Analysis and Design (OOAD) and in the use of various Design Patterns
• In-dept Understanding of Web Services and SOA concepts
• Experience with developing interoperability solutions for other technologies such as Java or Flash/Flex

Job Experience - Helpful Skills

• Experience with developing Flash/Flex Applications using ActionScript 2.0 and 3.0
• Exposure to Microsoft LINQ and The ADO.NET Entity Framework
• Experience in addressing security and transactions for Web Services
• MS SQL Server Admin and Performance Tuning Experience
• Experience in Monitoring and Tuning of Web Applications deployed on the Microsoft IIS Server versions 6.0 and 7.0
• Microsoft SQL Server Reporting & Analytics Services
• Experience with the use of the Microsoft ClickOnce Deployment Technology for distribution of application updates
• Exposure to Microsoft Azure Cloud Platform
• MCPD:Enterprise Application Developer Certification
• Exposure to Data Interchange Formats such as JSON in addition to XML and to understanding performance considerations arising from their use
• Exposure to use of various media types for video and still images in Desktop and Web Applications

Please note: We are seeking local Chicago Area Candidates not requiring sponsorship.
NO RECRUITERS PLEASE!!

Digital Aisle LLC is an Equal Opportunity Employer

I’ve been talking a lot about external lists lately as you may know.  They are so easy to create with SharePoint Designer, but you wouldn’t really deploy them to production that way would you?  Of course not!  We need a feature.  At first, you might think generating all of that CAML might be too difficult, but remember we can use the Save Site as Template page to generate a .wsp file for us.  Using this we can import it into Visual Studio and learn what CAML we need to use.

We’re going to start with the same external list from yesterday.  You can build your own using SharePoint Designer as described in my BCS post.

Once you are happy with your list, go to Site Settings –> Save Site as Template.  Fill in the fields and it will save a solution package in your solution gallery of your site collection.  Save the .wsp file to disk and we’re ready to begin.  Once you have your solution package, open Visual Studio 2010, create a new project, and use the Import SharePoint Solution Package project template.  You will be prompted for what site to use and where the package file is.  On the next step, you will be prompted for what you want to import.

The solution package has every site column, content type, page, list, etc, on the site so there is a lot to sift through.  However, the only thing we really need is the list instance in question (in my case Products).  You will want to unselect everything else as it will take forever to import if you don’t.

Quick Tip: There isn’t a select all button, so press Ctrl+A, and then unclick a checkbox to deselect everything.

On the next step you will get a warning about dependencies.  Go ahead and just say yes and your project will import.  Your project will likely have a bunch of stuff you don’t really care about.  All that you really need is what it is created in List Instances.  Your list instance might look something like this.

At this point, I created a new project because I want to keep things clean.  Create a new project and create a new List Definition.  At this point, you will have a schema file and two new elements files.  The elements.xml file with the list template can be deleted since I don’t want users creating new versions of this list.  Before we start digging into the CAML that is required to make an external list work, let’s go back and look at SharePoint Designer so we can remember what our application definition looked like.

The reason I show you this is because these values we’re going to see when we take a look at our elements.xml file.  Here is the file that came from the site export.

A couple of things to note here.  First, the TemplateType is 600.  We can only presume this is the list template id for an external list.  The other thing of note is the new DataSource element.  This was not in previous versions of SharePoint.  As you might notice here, these values correspond to what we see on the external content type.  This is what you will change should you decide to rename the entity in or change the LobSystemInstance name.  This XML is highly useable in our own feature, but I am going to remove some of the unnecessary attributes such as FeatureId and CustomSchema.  I’m also going to give it a new Title so that we know this is a different list.  Here is what my new elements.xml looks like.

Honestly, I think I can get rid of HyperLinkBaseUrl as well.  It doesn’t seem to matter though, I deployed it to a few other sites and it seems to work fine.  Now let’s take a look schema.xml.  It’s actually pretty small which is nice compared to the files we were used to in the past.

As you can see it’s pretty clean.  It again uses the Type of 600.  In the ContentTypes section, it does define a regular list item, but it doesn’t actually add any of the fields from the external list there.  The Fields element has a field defined for each field in my external content type.  As you can see there is nothing special about the way they are defined.  The only one of note is the BdcIdentity field which I assume is required to keep track of the ID that ties back to the BCS itself.  The View is surprisingly clean as well.  The XslLink and Method elements are new.  I assume we can use XslLink to customize how the view is rendered, but I didn’t actually see a main.xsl file generated in the solution package anywhere.  The Method element I can only assume corresponds to the name of the finder method Read List which we have seen above.  The rest is pretty simple.  It just has a FieldRef for each column in my external content type. 

I can pretty much use the Schema.xml file as is.  I did change the Title and Url attributes at the top but that is it.  At this point my new feature to deploy this is ready to go.  Here is what it looks like in Visual Studio.

Deploy your solution and have it activate your feature.  The new list won’t show up automatically in the navigation on the side, but it will be there.  Just go to the URL directly or view the lists on your site to get there.  Here it is on my new site.

As you can see it’s really not that hard to deploy an external list to another site properly using a feature.  Now, you might have noticed there are a few other elements.xml files generated when we did the site export.  One stores specific values in the property bag of the list.  This we did not want because we don’t really want to copy those internal properties.  The other deals with the forms of the list.  It uses the BinarySerializedWebPart which scares me a bit, so I haven’t messed with it much.  If you are just using default forms you don’t need to worry about it.  In a future post, I’ll try seeing if I can deploy some InfoPath forms along with the list, but I figured that deserves its own post.

Also, one other thing to note.  If you are deploying your external content type to another server, you can do that in the same manner as you did in SharePoint 2007.  Just export the application definition and import it on the new server.  You can also use the Export Application Model button in SharePoint designer on the External Content Types list.

I hope this helps when you look to move lists into production.  This technique of exporting and importing will also work with regular lists of course.  Just remember, friends don’t let friends deploy lists without a feature.

read more