Shard-Query is an open source tool kit which helps improve the performance of queries against a MySQL database by distributing the work over multiple machines and/or multiple cores. This is similar to the divide and conquer approach that Hive takes in combination with Hadoop. Shard-Query applies a clever approach to parallelism which allows it to significantly improve the performance of queries by spreading the work over all available compute resources. In this test, Shard-Query averages a nearly 6x (max over 10x) improvement over the baseline, as shown in the following graph: One significant advantage of Shard-Query over Hive is that it works with existing MySQL data sets and queries. Another advantage is that it works with all MySQL storage engines. This set of benchmarks evaluates how well Infobright community edition (ICE) performs in combination with Shard-Query. Data set It was important to choose a data set that was large enough to create queries that would run for a decent amount of time, but not so large that it was difficult to work with. The ontime flight performance statistics data, available online from the United States Bureau of Transportation Statistics (BTS) made a good candidate for testing, as it had been tested before: Another MPB post Lucid DB testing The raw data is a completely denormalized schema (single table). In order to demonstrate the power of Shard-Query it is important to test complex queries involving joins and aggregation. A star schema is the most common OLAP/DW data model, since it typically represents a data mart. See also: “Data mart or data warehouse?”. As it is the most common data model, it is desirable to benchmark using a star schema, even though it involves work to transform the data. Star schema Transforming the data was straightforward. I should note that I did this preprocessing with the MyISAM storage engine, then I dumped the data to tab delimited flat files using mysqldump. I started by loading the raw data from the BTS into a single database table called ontime_stage. Then, the airport information was extracted: create table dim_airport( airport_id int auto_increment primary key, unique key(airport_code) ) as select Origin as `airport_code`, OriginCityName as `CityName`, OriginState as `State`, OriginStateFips as `StateFips`, OriginStateName as `StateName` , OriginWac as `Wac` FROM ontime_stage UNION select Dest as `airport_code`, DestCityName as `CityName`, DestState as `State`, DestStateFips as `StateFips`, DestStateName as `StateName` , DestWac as `Wac` FROM ontime_stage; After extracting flight/airline and date information in a similar fashion, a final table `ontime_fact` is created by joining the newly constructed dimension table tables to the staging tables, omitting the dimension columns from the projection, instead replacing them with the dimension keys: select dim_date.date_id, origin.airport_id as origin_airport_id, dest.airport_id as dest_airport_id, dim_flight.flight_id, ontime_stage.TailNum, ... from ontime_stage join dim_date using(FlightDate) join dim_airport origin on ontime_stage.Origin = origin.airport_code join dim_airport dest on ontime_stage.Dest = dest.airport_code join dim_flight using (UniqueCarrier,AirlineID,Carrier,FlightNum); The data set contains ontime flight information for 22 years, which can be confirmed by examining the contents of the date dimension: mysql> select count(*), min(FlightDate), max(FlightDate) from dim_date\G *************************** 1. row *************************** count(*): 8401 min(FlightDate): 1988-01-01 max(FlightDate): 2010-12-31 1 row in set (0.00 sec) The airport dimension is a puppet dimension. It is called a puppet because it serves as both origin and destination dimensions, being referenced by origin_airport_id and destination_airport_id in the fact table, respectively. There are nearly 400 major airports included in the data set. mysql> select count(*) from dim_airport; +----------+ | count(*) | +----------+ | 396 | +----------+ 1 row in set (0.00 sec) The final dimension is the flight dimension, which contains the flight numbers and air carrier hierarchies. Only the largest air carriers must register and report ontime information with the FAA, so there are only 29 air carriers in the table: mysql> select count(*), count(distinct UniqueCarrier) from dim_flight\G *************************** 1. row *************************** count(*): 58625 count(distinct UniqueCarrier): 29 1 row in set (0.02 sec) Each year has tens of millions of flights: mysql> select count(*) from ontime_one.ontime_fact; +-----------+ | count(*) | +-----------+ | 135125787 | +-----------+ 1 row in set (0.00 sec) This should be made fully clear by the following schema diagram: Diagram of the ontime dimensional schema Test environment For this benchmark, a test environment consisting of a single commodity database server with 6 cores (+6ht) and 24GB of memory was selected. The selected operating system was Fedora 14. Oracle VirtualBox OSE was used to create six virtual machines, each running Fedora 14. Each of the virtual machines was granted 4GB of memory. A SATA 7200rpm RAID10 battery backed RAID array was used as the underlying storage for the virtual machines. Baseline: The MySQL command line client was used to execute the a script file containing the 11 queries. This same SQL script was used in the Shard-Query tests. For the baseline, the results and response times were captured with the \T command. The queries were executed on a single database schema containing all of the data. Before loading, there was approximately 23GB of data. After loading, ICE compressed this data to about 2GB. The test virtual machine was assigned 12 cores in this test. Scale-up: Shard-Query was given the following configuration file, which lists only one server. A single schema (ontime_one) contained all of the data. The test virtual machine was assigned 12 cores for this test. The same VM was used as the previous baseline test. This VM was rebooted between tests. A SQL script was fed to the run_query.php script and the output was captured with the ‘tee’ command. $ cat one.ini [default] user=mysql db=ontime_one password= port=5029 column=date_id mapper=hash gearman=127.0.0.1:7000 inlist=* between=* [shard1] host=192.168.56.102 Scale-out In addition to adding parallelism via scale-up, Shard-Query can improve performance of almost all queries by spreading them over more than one physical server. This is called “scaling out” and it allows Shard-Query to vastly improve the performance of queries which have to examine a large amount of data. Shard-Query includes a loader (loader.php) which can be used to either split a data into multiple files (for each shard, for later loading) or it can load files directly, in parallel, to multiple hosts. Shard-Query will execute queries in parallel over all of these machines. With enough machines, massive parallelism is possible and very large data sets may be processed. As each machine examines only a small subset of the data, performance can be improved significantly: $ cat shards.ini [default] user=mysql db=ontime password= port=5029 column=date_id mapper=hash gearman=127.0.0.1:7000 inlist=* between=* [shard1] host=192.168.56.101 db=ontime [shard2] host=192.168.56.102 db=ontime [shard3] host=192.168.56.103 db=ontime [shard4] host=192.168.56.105 db=ontime [shard5] host=192.168.56.106 db=ontime [shard6] host=192.168.56.104 db=ontime In this configuration, each shard has about 335MB-350MB of data (23GB raw data, compressed to about 2GB total data. then spread over six servers). Due to ICE limitations, the data was split before loading. The splitting/loading process will be described in another post. Complex queries Shard-Query was tested with the simple single table version of this data set in a previous blog post. Previous testing was limited to a subset of Vadim’s test queries (see that post). As this new test schema is normalized, Vadim’s test queries must be modified to reflect the more complex schema structure. For this benchmark each of the original queries has been rewritten to conform to the new schema, and additionally two new test queries have been added. To model real world complexity, each of the test queries feature at least one join, and many of the filter conditions are placed on attributes in the dimension tables. It will be very interesting to test these queries on various engines and databases. Following is a list of the queries, followed by a response time table recording the actual response times for each query. The queries should be self-explanatory. Performance, at a glance You will notice that Shard-Query is faster in nearly every case. The performance of the queries is improved significantly by scaling out, even more so than scaling up, because additional parallelism is added beyond what can be exploited by scale up. Scale up can improve query performance when there is enough resources to support the added parallelism, and when one of the the following are in used in the query: BETWEEN or IN clauses, subqueries in the FROM clause, UNION or UNION ALL clauses. If none of those features are used, then parallelism can’t be added. Q9 is an example of such a query. . Query details: -- Q1 SELECT DayOfWeek, count(*) AS c from ontime_fact JOIN dim_date using (date_id) WHERE Year BETWEEN 2000 AND 2008 GROUP BY DayOfWeek ORDER BY c DESC; -- Q2 SELECT DayOfWeek, count(*) AS c from ontime_fact JOIN dim_date using (date_id) WHERE DepDelay>10 AND Year BETWEEN 2000 AND 2008 GROUP BY DayOfWeek ORDER BY c DESC; -- Q3 SELECT CityName as Origin, count(*) AS c from ontime_fact JOIN dim_date using (date_id) JOIN dim_airport origin ON origin_airport_id = origin.airport_id WHERE DepDelay>10 AND Year BETWEEN 2000 AND 2008 GROUP BY 1 ORDER BY c LIMIT 10; The next queries show how performance is improved when Shard-Query adds parallelism when “subqueries in the from clause” are used. There are benefits with both “scale-up” and “scale-out”, but once again, the “scale-out” results are the most striking. -- Q4 SELECT Carrier, count(*) as c from ontime_fact JOIN dim_date using (date_id) join dim_flight using(flight_id) WHERE DepDelay>10 AND Year=2007 GROUP BY Carrier ORDER BY c DESC; -- Q5 SELECT t.Carrier, c, c2, c*1000/c2 as c3 FROM (SELECT Carrier, count(*) AS c from ontime_fact join dim_date using(date_id) join dim_flight using(flight_id) WHERE DepDelay>10 AND Year=2007 GROUP BY Carrier) t JOIN (SELECT Carrier, count(*) AS c2 from ontime_fact join dim_date using(date_id) join dim_flight using(flight_id) WHERE Year=2007 GROUP BY Carrier) t2 ON (t.Carrier=t2.Carrier) ORDER BY c3 DESC; -- Q6 SELECT t.Year, c1 / c2 as ratio FROM (select Year, count(*)*1000 as c1 from ontime_fact join dim_date using (date_id) WHERE DepDelay>10 GROUP BY Year) t JOIN (select Year, count(*) as c2 from ontime_fact join dim_date using (date_id) WHERE DepDelay>10 GROUP BY Year) t2 ON (t.Year=t2.Year); -- Q7 SELECT t.Year, c1 / c2 as ratio FROM (select Year, count(Year)*1000 as c1 from ontime_fact join dim_date using (date_id) WHERE DepDelay>10 GROUP BY Year) t JOIN (select Year, count(*) as c2 from ontime_fact join dim_date using (date_id) GROUP BY Year) t2 ON (t.Year=t2.Year); The performance of the following queries depends on the size of the date range: -- Q8.0 SELECT dest.CityName, COUNT( DISTINCT origin.CityName) from ontime_fact JOIN dim_airport dest on ( dest_airport_id = dest.airport_id) JOIN dim_airport origin on ( origin_airport_id = origin.airport_id) JOIN dim_date using (date_id) WHERE Year BETWEEN 2001 and 2001 GROUP BY dest.CityName ORDER BY 2 DESC; -- Q8.1 SELECT dest.CityName, COUNT( DISTINCT origin.CityName) from ontime_fact JOIN dim_airport dest on ( dest_airport_id = dest.airport_id) JOIN dim_airport origin on ( origin_airport_id = origin.airport_id) JOIN dim_date using (date_id) WHERE Year BETWEEN 2001 and 2005 GROUP BY dest.CityName ORDER BY 2 DESC; -- Q8.2 SELECT dest.CityName, COUNT( DISTINCT origin.CityName) from ontime_fact JOIN dim_airport dest on ( dest_airport_id = dest.airport_id) JOIN dim_airport origin on ( origin_airport_id = origin.airport_id) JOIN dim_date using (date_id) WHERE Year BETWEEN 2001 and 2011 GROUP BY dest.CityName ORDER BY 2 DESC; -- Q8.3 SELECT dest.CityName, COUNT( DISTINCT origin.CityName) from ontime_fact JOIN dim_airport dest on ( dest_airport_id = dest.airport_id) JOIN dim_airport origin on ( origin_airport_id = origin.airport_id) JOIN dim_date using (date_id) WHERE Year BETWEEN 1990 and 2011 GROUP BY dest.CityName ORDER BY 2 DESC; -- Q8.4 SELECT dest.CityName, COUNT( DISTINCT origin.CityName) from ontime_fact JOIN dim_airport dest on ( dest_airport_id = dest.airport_id) JOIN dim_airport origin on ( origin_airport_id = origin.airport_id) JOIN dim_date using (date_id) WHERE Year BETWEEN 1980 and 2011 GROUP BY dest.CityName ORDER BY 2 DESC; Finally, Shard-Query performance continues to improve when grouping and filtering is used. Again, notice Q9. It doesn’t use any features which Shard-Query can use to add parallelism. Thus, in the scale up configuration it does not perform any better than the baseline, and actually performed just a little worse. Since scale out splits the data between servers, it performs about 6x better as the degree of parallelism is controlled by the number of shards. -- Q9 select Year ,count(Year) as c1 from ontime_fact JOIN dim_date using (date_id) group by Year; -- Q10 SELECT Carrier, dest.CityName, COUNT( DISTINCT origin.CityName) from ontime_fact JOIN dim_airport dest on ( dest_airport_id = dest.airport_id) JOIN dim_airport origin on ( origin_airport_id = origin.airport_id) JOIN dim_date using (date_id) JOIN dim_flight using (flight_id) WHERE Year BETWEEN 2009 and 2011 GROUP BY Carrier,dest.CityName ORDER BY 3 DESC; -- Q11 SELECT Year, Carrier, dest.CityName, COUNT( DISTINCT origin.CityName) from ontime_fact JOIN dim_airport dest on ( dest_airport_id = dest.airport_id) JOIN dim_airport origin on ( origin_airport_id = origin.airport_id) JOIN dim_date using (date_id) JOIN dim_flight using (flight_id) WHERE Year BETWEEN 2000 and 2003 AND Carrier = 'AA' GROUP BY Year, Carrier,dest.CityName ORDER BY 4 DESC; Conclusion The divide and conquer approach is very useful when working with large quantities of data. Shard-Query can be used with existing data sets easily, improving the performance of queries significantly if they use common query features like BETWEEN or IN. It is also possible to spread your data over multiple machines, scaling out to improve query response times significantly. These queries are a great test of Shard-Query features. It is currently approaching RC status. If you decide to test it and encounter issues, please file a bug on the bug tracker. You can get Shard-Query (currently in development release form as a checkout from SVN) here: Shard-Query Google code project Full disclosure Justin Swanhart, the author of this article is also the creator and maintainer of Shard-Query. The author has previously worked in cooperation with Infobright, including on benchmarking. These particular tests were performed independently of Infobright, without their knowledge or approval. Infobright was, however, given the chance to review this document before publication, as a courtesy. All findings are represented truthfully, transparently, and without any intended bias.

Founded in 2002, Big Fish Games is a multi-platform developer, publisher and distributor of casual games, including downloadable, flash, social, MMO, console and mobile games. Renowned for offering A New Game Every Day!®, Big Fish Games distributes more than 1.5 million games per day worldwide. With headquarters in Seattle and offices in Vancouver, Canada, and Cork, Ireland, Big Fish Games partners with 500+ game developers and develops and publishes some of the industry's leading brands, including Mystery Case Files®, Hidden Expedition®, Drawn™, Fairway Solitaire™, and My Tribe™.

We’re seeking you: A creative, Senior DBA who will play a key role on the database team responsible for supporting our current MySQL infrastructure and design. This will include database design and architecture for existing and new initiatives, database monitoring, tuning, and capacity planning for production systems. This is a great opportunity to pull from your architecting and design experience to help us build cutting-edge and scalable solutions for social media applications.

Responsibilities include but are not limited:

• Working directly with developers on related database initiatives, and validating the scaling model and application performance
• Reviewing schema design and data definitions for production systems
• Evaluating and recommending database hardware and software
• Monitoring and performance tuning of production database systems
• Ensuring all maintenance, system integrity, capacity, scalability and space requirements for Big Fish Games database environments are met

Skills and experience you should be bringing to the table:

• 4+ years experience with databases in a production environment
• Previous experience high transaction and 50+ database server environments
• Knowledge of various database replication and clustering methods
• Solid understanding of major components in relational database architecture
• Experience in database backup, restore and disaster recovery procedures
• Experience with analyzing and tuning databases for performance and efficiency
• Skills in at least one of the following: Shell Scripting, Perl, Python or PHP
• Expert troubleshooting skills and abilities
• Ability to effectively interface and communicate with engineering teams
• Business savvy; an understanding of how our actions affect the business

Related experience and knowledge it would be nice to have:

• Experience with multiple database platforms
• Knowledge of and/or previous experience with Linux or UNIX
• Basic understanding of Java/JVM
• Bachelors degree in technology, engineering, or one with a mathematical focus

Come be the next Big Fish in the Pond!

To apply for this position, please visit our careers page at www.bigfishgames.com/careers and complete your online profile to upload your resume.

• Location: Seattle, WA
• Principals only. Recruiters, please don't contact this job poster.
• Please, no phone calls about this job!
• Please do not contact job poster about other services, products or commercial interests.

MySQL Lunch-n-Learn is not a new performance schema table or a new BLOB data type for Chef Gordon Ramsey. It is where a MySQL engineer comes to your office bringing lunch to talk about the most popular open source database. I was asked to participate in one this morning at a major company headquartered in Dallas. The company wanted some information on updates to the MySQL server, a comparison between InnoDB and MyISAM, and some company specific questions.It is often hard to get new product news to our users and this program provides us wit ha way to answer specific questions. And the customer gets to select the agenda items to make certain they get what they need form this event. So if you want to get the latest on MySQL 5.6, Memcahched-to-InnoDB, or replication strategies, use the link at the top of this entry to set up your Lunch-n-Learn.

MySQL, the most popular Open Source SQL database, is provided by MySQL AB. MySQL AB is a commercial company that builds its business providing services around the MySQL database.

Version 5.1.57:

Functionality added or changed:

• When invoked with the --auto-generate-sql option, mysqlslap dropped the schema specified with the --create-schema option at the end of the test run, which may have been unexpected by the user. mysqlslap no longer drops the schema, but has a new --create-and-drop schema that both creates and drops a schema. (Bug #58090, Bug #11765157)
• A new system variable, max_long_data_size, now controls the maximum size of parameter values that can be sent with the mysql_stmt_send_long_data() C API function. If not set at server startup, the default is the value of the max_allowed_packet system variable. This variable is deprecated. In MySQL 5.6, it is removed and the maximum parameter size is controlled by max_allowed_packet.

Bugs fixed:

• InnoDB Storage Engine: Replication: Trying to update a column, previously set to NULL, of an InnoDB table with no primary key caused replication to fail with Can't find record in 'table' on the slave. (Bug #11766865, Bug #60091)
• InnoDB Storage Engine: The server could halt if InnoDB interpreted a very heavy I/O load for 15 minutes or more as an indication that the server was hung. This change fixes the logic that measures how long InnoDB threads were waiting, which formerly could produce false positives. (Bug #11877216, Bug #11755413, Bug #47183)
• Replication: Using the --server-id option with mysqlbinlog could cause format description log events to be filtered out of the binary log, leaving mysqlbinlog unable to read the remainder of the log. Now such events are always read without regard to the value of this option.
• As part of the the fix for this problem, mysqlbinlog now also reads rotate log events without regard to the value of --server-id. (Bug #11766427, Bug #59530)
• Partitioning: A problem with a previous fix for poor performance of INSERT ON DUPLICATE KEY UPDATE statements on tables having many partitions caused the handler function for reading a row from a specific index to fail to store the ID of the partition last used. This caused some statements to fail with Can't find record errors. (Bug #59297, Bug #11766232)
• InnoDB invoked some zlib functions without proper initialization. (Bug #11849231)
• Selecting from a view for which the definition included a HAVING clause failed with an error:
• 1356: View '...' references invalid table(s) or column(s)or function(s) or definer/invoker of view lack rights to use them(Bug #60295, Bug #11829681)
• The server permitted max_allowed_packet to be set lower than net_buffer_length, which does not make sense because max_allowed_packet is the upper limit on net_buffer_length values. Now a warning occurs and the value remains unchanged. (Bug #59959, Bug #11766769)
• The server read one byte too many when trying to process an XML string lacking a closing quote (') or double quote (") character used as an argument for UpdateXML() or ExtractValue(). (Bug #59901, Bug #11766725)
• See also Bug #44332, Bug #11752979.
• Attempting to create a spatial index on a CHAR column longer than 31 bytes led to an assertion failure if the server was compiled with safemutex support. (Bug #59888, Bug #11766714)
• Aggregation followed by a subquery could produce an incorrect result. (Bug #59839, Bug #11766675)
• An incorrect character set pointer passed to my_strtoll10_mb2() caused an assertion to be raised. (Bug #59648, Bug #11766519)
• A missing variable initialization for Item_func_set_user_var objects could cause an assertion to be raised. (Bug #59527, Bug #11766424)
• mysqldump did not quote database names in ALTER DATABASE statements in its output, which could cause an error at reload time for database names containing a dash. (Bug #59398, Bug #11766310)
• In Item_func_month::val_str(), a Valgrind warning for a too-late NULL value check was corrected. (Bug #59166, Bug #11766126)
• In Item::get_date, a Valgrind warning for a missing NULL value check was corrected. (Bug #59164, Bug #11766124)
• In extract_date_time(), a Valgrind warning for a missing end-of-string check was corrected. (Bug #59151, Bug #11766112)
• In string context, the MIN() and MAX() functions did not take into account the unsignedness of a BIGINT UNSIGNED argument. (Bug #59132, Bug #11766094)
• In Item_func::val_decimal, a Valgrind warning for a missing NULL value check was corrected. (Bug #59125, Bug #11766087)
• In Item_func_str_to_date::val_str, a Valgrind warning for an uninitialized variable was corrected. (Bug #58154, Bug #11765216)
• The code for PROCEDURE ANALYSE() had a missing DBUG_RETURN statement, which could cause a server crash in debug builds. (Bug #58140, Bug #11765202)
• An assertion could be raised in Item_func_int_val::fix_num_length_and_dec() due to overflow for geometry functions. (Bug #57900, Bug #11764994)
• An assertion could be raised if a statement that required a name lock on a table (for example, DROP TRIGGER) executed concurrently with an INFORMATION_SCHEMA query that also used the table. (Bug #56541, Bug #11763784)
• For a client connected using SSL, the Ssl_cipher_list status variable was empty and did not show the possible cipher types. (Bug #52596, Bug #11760210)
• With lower_case_table_names=2, resolution of objects qualified by database names could fail. (Bug #50924, Bug #11758687)
• A potential invalid memory access discovered by Valgrind was fixed. (Bug #48053, Bug #11756169)
• Bitmap functions used in one thread could change bitmaps used by other threads, causing an assertion to be raised. (Bug #43152, Bug #11752069)
• SHOW EVENTS did not always show events from the correct database. (Bug #41907, Bug #11751148)

Mac OS X 10.4 or later

Download Now

Last month was one of the busiest months for the InnoDB team: MySQL 5.6.2 milestone release with many InnoDB features MySQL Lab release of NoSQL to InnoDB with Memcached Three talks at O’Reilly MySQL Conference and COLLABORATE 11 Ten blogs The slides of the three talks are now available: InnoDB: Status, Architecture, and Latest Enhancements, O’Reilly MySQL Conference, April 14, 2011, by Inaam Rana, John Russell InnoDB: Performance and Scalability Features, COLLABORATE 11, April 11, 2011, by Calvin Sun InnoDB: Status, Architecture, and Latest Enhancements, COLLABORATE 11, April 11, 2011, by Calvin Sun In addition to the blogs from the InnoDB team, Dimitri published a five-part series on MySQL 5.6 Performance, all on DimitriK’s (dim) Weblog: MySQL Performance: 5.6 Notes, part 5 – Fixing Adaptive Flushing.. MySQL Performance: 5.6 Notes, part 4 – Fixing Purge issue.. MySQL Performance: 5.6 Notes, part 3 – More in depth.. MySQL Performance: 5.6 Notes, part 2 – Under full dbSTRESS workload… MySQL Performance: 5.6 Notes, part 1 – Discovery… As usual, Dimitri’s blogs are full of nice charts, detail analysis, and sometimes critiques. In part 3, he talked about his experience on InnoDB METRICS table, a new feature in MySQL 5.6.2 release. Øystein Grøvlen, a member of MySQL optimizer team, published an article on InnoDB Persistent Statistic: InnoDB Persistent Statistics Save the Day, also a new feature in MySQL 5.6.2 release. The blogs written by InnoDB team: MySQL 5.6: Multi threaded purge, by Sunny Bains Tips and Tricks for Faster DDL, by Marko Mäkelä MySQL 5.6: Data dictionary LRU, by Sunny Bains Introducing page_cleaner thread in InnoDB, by Inaam Rana Information Schema for InnoDB System Tables, by Jimmy Yang InnoDB Persistent Statistics at last, by Vasil Dimov MySQL 5.6: InnoDB scalability fix – Kernel mutex removed, by Sunny Bains NoSQL to InnoDB with Memcached, by Calvin Sun Get started with InnoDB Memcached Daemon plugin, by Jimmy Yang Get started with InnoDB Metrics Table, by Jimmy Yang

''Necessario intensificare la pressione per far partire l'iniziativa politica''
Libia, Frattini: ''Stop raid in 3-4 settimane'' Nato: ''Avanti fino a raggiungere obiettivi''


(06/05/2011 - 18:39:57)

Recognized by Zagat Survey as New York City's most popular off-price retailer, Century 21 Department Stores continues to recruit talented professionals that are dedicated to high standards of excellence. Our organization values people who are dedicated and in search of a rewarding career, not just a job. Century 21 Department Store’s first-class approach to retail makes being a part of Century 21 Department Stores a truly rewarding experience.

Century 21 Department Stores, established in 1961, is legendary for its exceptional offering of designer label and brand name merchandise sold at unbelievable prices. A quality fashion leader for over 40 years, Century 21 Department Stores is known for its trendsetting styles and first quality assortment of merchandise.

From the newest looks to the timeless classics, from casual to career, from day into night, Century 21 Department stores dresses the entire family in the top names in fashion at discount prices.

Century 21 Department stores is headquartered in New York City and operates 6 stores, including: Lower Manhattan; Brooklyn; Westbury, Long Island; Morristown, New Jersey, Paramus New Jersey and Rego Park, Queens. We are planning to open up our newest store in the fall of 2011 on the Upper Westside of Manhattan.

If you are; loyal, passionate, innovative, ambitious, customer focused, and ready to make a unique contribution, come join a company that knows success and is growing in a purposeful way. This company and job will challenge you to use your talents and become something more (being alive everyday; versus a boring job)

We are currently recruiting for the following positon based out of our Secaucus, New Jersey offices:

Post Production Photo Editor. This position will report to the Web Production Manager and is a wonderful opportunity for a recent college grad.


Overall Responsibilities:

Member of the Web Production Team
Retouch images for the website
Maintain accurate color temperature, contrast and image quality for image consistency across the website
Rename and save images based upon prescribed schema
Upload images to website
Archive all photography
Update info in Merchandise Tracking System
Participate in monthly review of front end of the website for usability continuity
Utilize best practices and embrace future enhancements of digital technologies


Skillset & Experience:

1-2 year retouching and photography experience
Adobe Photoshop
Photography background preferred
HTML – basic knowledge of creating tables and image maps
MAC and PC proficient
Creative problem solving skills

To apply go to c21stores.com and click on the career section. Then click on Corporate Postions and apply to the Post Production Photo Editor Position. The application process takes between 10 minutes to complete. Please include your salary expectations in your cover letter if you want to be considered for this position!!!!

• Location: Secaucus, NJ
• Compensation: We are offering a compeititve salary including all benefits!!!!
• Principals only. Recruiters, please don't contact this job poster.
• Please, no phone calls about this job!
• Please do not contact job poster about other services, products or commercial interests.

Rapidly growing Seattle area rich media production company looking for a passionate and highly skilled LAMP development veteran.

Responsibilities will include: Database and back-end design for a media rich content delivery system including but not limited to: content and user management systems, metrics and reporting systems, content browsing and delivery systems. Working collaboratively with UI/UX designers that share a similar passion for perfection. Being an integral part of the team responsible for showcasing our industry-leading production content.

Skills
Must Have:
5+ years of programming experience working on more than one project, at least one of which is a successful consumer-oriented product that you supported for at least one year
Proficient in PHP, MySQL, MemCache (or similar NoSQL experience) and MVC (or similar design pattern experience)
Nice to have any of these:
Amazon AWS experience or equivalent cloud experience
Jquery and/or AJAX
Experience with distributed MySQL Databases
Schema optimization and/or Database tuning
LAMP Security
LAMP / Linux Administration
Horizontal scaling experience


We don't abide the usual product development process and neither does our team. Can't stop working until the results are perfect? Interested in owning the ground-up design and build of a highly custom media delivery website? Enjoy flexible work environments and hours? You're clearly made for this job.

To Apply: Please email your resume and contact information to the craigslist email and we will contact you to setup an interview.

Pay is directly proportional to ability. Highly skilled candidates may end up VERY well compensated. Please, no agencies or overseas contractors.

• Location: Seattle
• Compensation: Pay is directly proportional to ability. Highly skilled candidates may end up VERY well compensated. Please, no agencies or overseas contractors.
• Telecommuting is ok.
• This is a part-time job.
• This is a contract job.
• Principals only. Recruiters, please don't contact this job poster.
• Please, no phone calls about this job!
• Please do not contact job poster about other services, products or commercial interests.

As you know, Tomas announced the availability of MySQL 5.6.2 Dev Milestone in his MySQL O'Reilly and Collaborate keynote presentations a few weeks back.  Downloads so far have been great, which reflects the feedback and excitement we picked up on from the attendees of both events, especially at the O'Reilly Community reception on that Wednesday (it was great to so many friends, family, peers there!).You can read John Russell's excellent recap of the new things in 5.6.2 and new innovations currently available in the MySQL Labs here.  And all of the 5.6 ready docs are available to help you learn how to enable and implement those you might find most interesting.I am especially excited about our continued use of the MySQL Labs to introduce new, development ready, atomic features "early and often" to our Community.  Although there has been some noise on the airways about the speed at which we deliver new features under Oracle our commitment to being the early, cutting edge technology leader in all things MySQL and InnoDB remains intact.  We will continue to use the MySQL Labs to quickly deliver the best MySQL and InnoDB core features and innovation to the best, most active open source Community on the planet.  Going forward, look for more Labs innovation around the new NotOnlySQL Memcached API for InnoDB and Cluster, Performance Schema, Replication and more.  MySQL 5.6.2 downloads are here to get you started, and you can get all of the Labs downloads here.As always, thanks for your continued support of the best and most widely used open source database in the world!  Go MySQL!

''Il Pd non riesce a essere sinistra''
Pd, Mario Adinolfi: ''Il Partito democratico non attrae i giovani''


(06/05/2011 - 13:57:36)

Why We Chose MongoDB for Guardian.co.uk (SlideShare) -- they're using MongoDB's flexible schema, as schema upgrades were pain in their previous system (Oracle). I think of these as the database equivalent of dynamic typing in languages like Perl and Ruby. (via Paul Rowe) Solving Problems with Visual Analytics -- This book is the result of a community effort of...

Summary: Learn how to use Windows PowerShell to quickly and efficiently copy a table between two SQL Server instances.

  Hey, Scripting Guy! How can I copy a table between two SQL Server instances?

—PY

Hello PY, Microsoft Scripting Guy, Ed Wilson, is here. Today is the last day for this week’s SQL Guest Blogger series. Today our guest is Yan Pan.

Yan Pan has 10 years working on various versions of SQL Server (MCITP SQL Server 2008, MCITP SQL Server 2005, MCDBA SQL Server 2000, plus OCA Oracle 10g), mainly in the financial services sector. She is a senior database architect, and she wrote a Wrox book titled Microsoft SQL Server 2008 Administration with Windows PowerShell. Besides SQL Server and Windows PowerShell, she also develops complex BI solutions with Microsoft technologies, works on other databases, such as Oracle and columnar databases, programs in .NET, and performs financial data development and research. If you would like to read more articles by her, visit her website at http://yanpansql.com/.

Very often, a DBA gets user requests to copy a table from a production to a development environment for testing purposes, or to synchronize a table in a reporting database with a production database. As a DBA, you can create an SSIS package with a Data Flow task in Business Intelligence Development Studio (BIDS), or you can launch the Import and Export Wizard from SQL Server Management Studio and let the wizard create an SSIS package for you.

However, you cannot reuse the same SSIS package to copy a different table with a different structure. You have to create a new package for the new table or modify the column mappings of an existing package in BIDS. It would be far more convenient to have a script that takes the names of the source server, source database, source table, destination server, destination database, and destination table as the parameters, and can be reused by simply providing different names.

Scripting can make a DBA’s life much easier. You might ask, “Why not use the bcp utility?” Yes, you can export the source table to a file and then import it to the destination table from the file. However, it requires creating a file on disk. We all know I/O is typically the most expensive resource with data manipulation activities. Of course, you can also use non-Microsoft tools, such as SQL Data Compare from Red Gate, but these tools come at a cost, and they are not usually optimized for your special use cases. They can be slow and resource intensive when the target table size is big.

Here I introduce a Windows PowerShell script that can be reused to copy the data from a source table to a destination table. This script uses the .NET SqlBulkCopy class with a streaming IDataReader to achieve the optimal performance (using a single thread).

Let us first define the parameters in the following param block. As you can see, the script requires a source server name, a source database name, a source table name, and a destination server name. The names of the destination database and table are optional. If you do not provide them, the script assumes that they are the same as the source. If you have a table in a non-default schema, you also need to specify the table names as two-part names. For example, if the user account under which you run the script maps to a user in the destination database with the default schema dbo, but you want to copy the data into an Order table in a non-default schema Test, then you need to specify the destination table name as Test.Order. The script also uses a switch parameter, Truncate. When this switch is included, the destination table is truncated and the existing data is removed before the copy.

Param (

      [parameter(Mandatory = $true)]

      [string] $SrcServer,

      [parameter(Mandatory = $true)]

      [string] $SrcDatabase,

      [parameter(Mandatory = $true)]

      [string] $SrcTable,

      [parameter(Mandatory = $true)]

      [string] $DestServer,

      [string] $DestDatabase, # Name of the destination database is optional. When omitted, it is set to the source database name.

      [string] $DestTable, # Name of the destination table is optional. When omitted, it is set to the source table name.

      [switch] $Truncate # Include this switch to truncate the destination table before the copy.

  )

 

  If ($DestDatabase.Length –eq 0) {

    $DestDatabase = $SrcDatabase

  }

 

  If ($DestTable.Length –eq 0) {

    $DestTable = $SrcTable

  }

 

  If ($Truncate) {

    $TruncateSql = "TRUNCATE TABLE " + $DestTable

    Sqlcmd -S $DestServer -d $DestDatabase -Q $TruncateSql

  }

Next, we prepare a SqlDataReader, a forward-only stream of rows from the source table. In our example, we have an Order table in the AdventureWorks database on the source server, YYCenter. Notice that we also create the helper function ConnectionString to return a connection string based on a server name and a database name.

Function ConnectionString([string] $ServerName, [string] $DbName)

  {

    "Data Source=$ServerName;Initial Catalog=$DbName;Integrated Security=True;"

  }

 

  $SrcConnStr = ConnectionString $SrcServer $SrcDatabase

  $SrcConn  = New-Object System.Data.SqlClient.SQLConnection($SrcConnStr)

  $CmdText = "SELECT * FROM " + $SrcTable

  $SqlCommand = New-Object system.Data.SqlClient.SqlCommand($CmdText, $SrcConn) 

  $SrcConn.Open()

  [System.Data.SqlClient.SqlDataReader] $SqlReader = $SqlCommand.ExecuteReader()

We are now ready to do the bulk insert into the destination table.

Try

  {

    $DestConnStr = ConnectionString $DestServer $DestDatabase

    $bulkCopy = New-Object Data.SqlClient.SqlBulkCopy($DestConnStr, [System.Data.SqlClient.SqlBulkCopyOptions]::KeepIdentity)

    $bulkCopy.DestinationTableName = $DestTable

    $bulkCopy.WriteToServer($sqlReader)

  }

  Catch [System.Exception]

  {

    $ex = $_.Exception

    Write-Host $ex.Message

  }

  Finally

  {

    Write-Host "Table $SrcTable in $SrcDatabase database on $SrcServer has been copied to table $DestTable in $DestDatabase database on $DestServer"

    $SqlReader.close()

    $SrcConn.Close()

    $SrcConn.Dispose()

    $bulkCopy.Close()

  }

You can also set a batch size, for example, 1000 rows, or increase the default time-out of 30 seconds to 60 seconds by setting the properties of the SqlBulkCopy instance as shown here.

 

$bulkCopy.BatchSize = 1000

$bulkCopy.BulkCopyTimeout = 0

Since Try/Catch/Finally block is only available in Windows PowerShell 2.0, we add the version tag into the script.

#requires -version 2.0

Let us put all the pieces of the script together. Here is the complete script Copy-SqlTable.ps1, and it can also be found in the Script Repository.

  #requires -version 2.0 

  Param (

      [parameter(Mandatory = $true)]

      [string] $SrcServer,

      [parameter(Mandatory = $true)]

      [string] $SrcDatabase,

      [parameter(Mandatory = $true)]

      [string] $SrcTable,

      [parameter(Mandatory = $true)]

      [string] $DestServer,

      [string] $DestDatabase, # Name of the destination database is optional. When omitted, it is set to the source database name.

      [string] $DestTable, # Name of the destination table is optional. When omitted, it is set to the source table name.

      [switch] $Truncate # Include this switch to truncate the destination table before the copy.

  )

 

  Function ConnectionString([string] $ServerName, [string] $DbName)

  {

    "Data Source=$ServerName;Initial Catalog=$DbName;Integrated Security=True;"

  }

 

  ########## Main body ############

  If ($DestDatabase.Length –eq 0) {

    $DestDatabase = $SrcDatabase

  }

 

  If ($DestTable.Length –eq 0) {

    $DestTable = $SrcTable

  }

 

  If ($Truncate) {

    $TruncateSql = "TRUNCATE TABLE " + $DestTable

    Sqlcmd -S $DestServer -d $DestDatabase -Q $TruncateSql

  }

 

  $SrcConnStr = ConnectionString $SrcServer $SrcDatabase

  $SrcConn  = New-Object System.Data.SqlClient.SQLConnection($SrcConnStr)

  $CmdText = "SELECT * FROM " + $SrcTable

  $SqlCommand = New-Object system.Data.SqlClient.SqlCommand($CmdText, $SrcConn) 

  $SrcConn.Open()

  [System.Data.SqlClient.SqlDataReader] $SqlReader = $SqlCommand.ExecuteReader()

 

  Try

  {

    $DestConnStr = ConnectionString $DestServer $DestDatabase

    $bulkCopy = New-Object Data.SqlClient.SqlBulkCopy($DestConnStr, [System.Data.SqlClient.SqlBulkCopyOptions]::KeepIdentity)

    $bulkCopy.DestinationTableName = $DestTable

    $bulkCopy.WriteToServer($sqlReader)

  }

  Catch [System.Exception]

  {

    $ex = $_.Exception

    Write-Host $ex.Message

  }

  Finally

  {

    Write-Host "Table $SrcTable in $SrcDatabase database on $SrcServer has been copied to table $DestTable in $DestDatabase database on $DestServer"

    $SqlReader.close()

    $SrcConn.Close()

    $SrcConn.Dispose()

    $bulkCopy.Close()

  }

I used the script to copy the Production.ProductInventory table in the AdventureWorks sample database on YYCenter to the Production.ProductInventory table in the AdventureWorks database on YYCenter\SQL2008R2. The following image shows the output of the script on my computer.

Thanks, Yan Pan—that was an excellent article.

Well, this concludes SQL Guest Blogger Week. Join me tomorrow for the Weekend Scripter as I kick off the 2011 Scripting Games wrap-up articles.

I invite you to follow me on Twitter and Facebook. If you have any questions, send email to me at scripter@microsoft.com, or post your questions on the Official Scripting Guys Forum. See you tomorrow. Until then, peace.

Ed Wilson, Microsoft Scripting Guy

Earlier this week I blogged about the new ASP.NET MVC 3 Tools Update that we shipped last month. 

In today’s blog post I’m going to go into more detail about two of the cool new features it brings:

1. Built-in support for EF 4.1 (which includes the new EF “code-first” support)
2. Built-in data scaffolding support within Visual Studio (which enables you to rapidly create data-driven sites)

These two features provide a really sweet, and extremely powerful, way to work with data and build data-driven web applications.

Scenario We’ll Build

To help illustrate how to use the above features, we’ll walkthrough building a simple data-drive site.  It will support listing products:

As well as creating/editing new products (and categories):

We can now build this entire application (and create the database that backs it) with ASP.NET MVC 3 in only a minute or two.

Step 1: Create New Project

We’ll begin by creating a new ASP.NET MVC 3 project (using the File->New Project menu command).  We’ll use the “Internet Project” template – which will provide us with a default starter template for our application. 

When you look at the newly created project within the Solution Explorer, you’ll see that the ASP.NET MVC 3 Tools update adds a new assembly – EntityFramework – to our ASP.NET MVC 3 projects:

The EntityFramework assembly implements Entity Framework 4.1 – which we also released last month.  EF 4.1 brings with it a bunch of great data API improvements to .NET – including the new “code first” option.  EF Code First provides a really elegant and clean way to work with data, and enables you to do so without requiring a designer or XML mapping file.  You can now easily take advantage of this by default with all new ASP.NET MVC 3 projects.

We’ll be using the EF Code First approach to implement our data access for this application.

Step 2: Implement Model Classes

Our first step will be to create two classes – Product and Category – that we’ll use to define the “model” of our application.  We’ll implement these as standard “plain old C# objects” within the \Models folder of our project using the code below:

Notice how the above classes are just standard classes with .NET data-types.  They do not need to derive from any base class, nor implement any interfaces. 

In addition to standard scalar properties, each class has an “association” property.  For example, the “Product” class has a “Category” property that enables a developer to get access to the Category it belongs to.  Likewise the “Category” class has a “Products” property that enables a developer to retrieve the Products that are within that particular Category.  EF Code First can handle automatically managing these associations (using the Primary Key/Foreign Key values), and can lazy load the appropriate data on-demand.

Step 3: Implement a StoreContext Class using EF Code First

Now that we’ve defined our two model classes, our next step will be to implement a “DbContext” class using EF code First that will map the model classes to tables in a database.  We’ll implement this using the code below:

The “StoreContext” class above is the DbContext class we are using to map our Product and Category classes to/from a database.  It derives from the DbContext base class provided by EF Code First, and exposes two properties that correspond to tables within our database.  For this sample we are using the default “convention over configuration” based mapping rules to define how the classes should map to/from the database.  This means that the Products property will map to a Products table, and the Categories property will map to a Categories table.  In later blog posts I’ll talk about how you can optionally override this and perform custom mappings.

You can add the above class anywhere within your solution.  For example, you could add it within the \Models folder of your project – or within a separate class library project if you prefer to keep your data code separate.  You’ll want to add a “using” statement to the System.Data.Entity namespace at the top of the class file (in order to reference the DbContext and DbSet<T> classes).

Step 4: Scaffold a Categories Controller

We have everything we need to model our data and save/retrieve it from a database.  Now let’s create an ASP.NET MVC Controller class that will implement the support necessary to display our Category data and enable creating/editing/deleting Category objects.  Prior to last month’s ASP.NET MVC 3 Tools Update, you would have had to manually write a Controller class to do this (and implement all of the EF data access code yourself).  The ASP.NET MVC 3 Tools Update now includes built-in “scaffolding” support that can help do this for you automatically.

To scaffold a new Categories controller class, we’ll right-click on the “/Controllers” folder of our project and choose the Add->Controller context menu:

Selecting Add->Controller will bring up the “Add Controller” dialog.  We’ll name the controller class we want to create “CategoriesController”.  Starting with last month’s ASP.NET MVC 3 Tools Update, the “Add Controller” dialog also now supports a new “Scaffolding options” section – which enables you to choose to “scaffold” the new controller using a variety of built-in templates:

We’ll choose the “Controller with read/write actions and views, using Entity Framework” template – which will create a Controller class that has all of the EF code necessary to create, update, list and delete model objects.

After selecting this template, we can choose the model class we want our Controller to implement CRUD support for – in this case the “Category” class.  We can also select the EF DbContext/ObjectContext class that we want to use to access the database – in this case the “StoreContext” class we defined earlier:

When we click the “Add” button, Visual Studio will automatically create a CategoriesController class for us – with appropriate Create, Edit, Details, Delete and Index action methods inside it – as well associated view templates for each of the actions:

If you open the CategoriesController.cs class you’ll find that the action methods within it implement the EF data access code necessary for CRUD support:

And now we have all of the code we need to implement data listing/editing for Categories within our database

Step 5: Configuring our Database Location

The last step we’ll do before running our application is to point our application at the database we want to use.  EF code first allows you to use an existing database you already have defined, or alternatively you can point it at a database that doesn’t yet exist – in which case it will automatically create it for you using the model classes you’ve defined.

No Connection-String

By default, you actually don’t need to configure a connection-string to point at a database.  If you don’t specify one, EF Code-First will by default create a new SQL Express database whose name matches your DbContext class name (for example: above it would create a ScaffoldingDemo.Models.StoreContext database within your local .\SQLExpress database instance). 

Explicitly Specifying a Connection-String

You can alternatively explicitly specify where you want your database to live by adding a connection-string to your application’s web.config file.  You’ll want the name of the connection-string to match the DbContext class name.  For example – in our application above our DbContext class was named “StoreContext”, and so by default EF will look for a connection-string with the same name to determine the database location.

Below is the entry we’d add to our web.config file if we wanted to use an embedded SQL CE database named “Store.sdf” within our application’s \App_data folder:

<configuration>
      <connectionStrings>
              <add name="StoreContext"
                         connectionString="Data Source=|DataDirectory|\Store.sdf"
                         providerName="System.Data.SqlServerCe.4.0" />
       </connectionStrings>

</configuration>

You can learn more about SQL CE – and the new VS 2010 SP1 tooling support for it – from my previous blog post about it.

Step 6: Run the Application

Ok – let’s now run the application and navigate to the /Categories URL.  This will execute our CategoriesController’s Index() method – which will list all categories in our database.  Currently we don’t have any:

Let’s click the “Create New” link to create a new category – this will navigate to the /Categories/Create URL, which provides us with a form to create a new category:

Clicking the “Create” button will post the form to our CategoriesController and add a new Category to our database.  We can repeat this a few times to add several different categories:

The cool thing is that we didn’t have to write any Controller or View code to enable this – the scaffolding support automatically generated the code we needed to enable this.  This provides a really easy way to get started.  We can then go in and tweak it as necessary to customize the functionality further.

Step 7: Looking at the database

We configured our application to use a connection-string that pointed to a SQL CE database that (at the time) didn’t exist.  When we ran our application, EF 4.1 detected that the database didn’t exist and automatically created it for us – based on the Product and Category classes we defined earlier. 

To see the database, click the “Show All Files” icon at the top of Visual Studio’s “Solution Explorer” and add the “Store.sdf” database that was automatically created by EF to your project:

You can then double-click the database to open it up in the Server Explorer – and expand the tables within it to see the schema that was created:

Notice above how EF Code First automatically created the appropriate database schema based on the Category and Product classes we created earlier.  It also automatically setup Primary Key/Foreign-Key relationships between the two tables based on the association properties specified in the classes.

Step 8: Scaffolding a Products Controller

We’ve created a CategoriesController class to manage listing and editing Categories.  Let’s now create a ProductsController class to manage listing and editing Products.

Just like before, we’ll right-click on the \Controllers folder in our project and choose the Add->Controller context menu.  We’ll name the Controller class we want to create “ProductsController” and indicate that we want to scaffold using the EF template with a Product model class:

Clicking the “Add” button will scaffold a ProductsController class for us, with appropriate action methods for CRUD scenarios, and create the associated view templates that go with them:

We can then run our project again and navigate to the /Products URL within it:

Let’s click the “Create New” link to create a new Product:

One cool thing to notice is that the “Category” is a drop-down of valid Categories (and not just a textbox with a foreign-key integer value):

The scaffolding support within Visual Studio was smart enough to detect that Category was an association, and scaffold EF code that populates the drop-down with valid Categories.

After entering several products our /Products URL will display the data like below:

Notice again how Category is displayed using the friendly name – and not just the foreign-key value.

Learning More

The above walkthrough provided a quick look at the new EF code-first and data scaffolding support built-into the ASP.NET MVC 3 Tools Update, and how you can use them to easily and quickly jumpstart application development.  EF Code First enables you to write incredibly clean data access code.  The data scaffolding support enables you to rapidly create data UI – and then allows you to go in and easily tweak/customize it further.

In addition to working with EF Code-First, you can also use the data scaffolding support against standard EF ObjectContext objects (built using either the database-first or model-first approach and the EF WYSIWYG designer).

Scaffolding

You can learn more about the new ASP.NET MVC 3 Scaffolding Support from Steve Sanderson’s excellent talk about it at MIX:

• Scaffolding – ASP.NET, NuGet, MVCScaffolding – Steve Sanderon’s MIX talk about it

In his talk Steve also covers how you can use the scaffolding support from the command-line (using NuGet) as well as how you can download and use additional scaffolding templates (for example: ones that can automatically implement a repository pattern, etc).  Also make sure you bookmark Steve’s excellent blog series about scaffolding (and its extensibility) here:

• Scaffolding with ASP.NET MVC 3 – Excellent blog series about ASP.NET MVC 3 Scaffolding

EF Code First

I’ll be doing more blog posts about EF Code First in the future.  Below are links to some tutorials I’ve written in the past about it:

• Code First Development with Entity Framework 4.x
• EF Code First: Custom Database Schema Mapping
• Using EF Code First with an Existing Database

Note: The above tutorials were written against the CTP4 release of EF Code First (and so some APIs are a little different) – but the concepts and scenarios outlined in them are the same as with the final release.

ASP.NET MVC 3

If you want to learn more about ASP.NET MVC 3, I highly recommend Scott Hanselman’s excellent video presentations at the Dutch DevDays and at MIX:

• ASP.NET MVC 3 101 – From Beginning to Advanced
• Overview of the MS Web Stack of Love – Scott’s talk at MIX

Also check out all of the new content and videos on the http://asp.net/mvc web-site:

• ASP.NET MVC 3 Videos and Tutorials

Summary

The new ASP.NET MVC 3 Tooling Update is full of goodness.  If you haven’t already downloaded and installed it I highly recommend you do so.  Stay tuned for even more blog posts about it!

Hope this helps,

Scott

P.S. I am also now using Twitter for quick updates and to share links. Follow me at: twitter.com/scottgu

Many development teams are used to making heavy use of branches in version control. Distributed version control systems make this even more convenient. Thus one of the more controversial statements in Continuous Delivery is that you can’t do continuous integration and use branches. By definition, if you have code sitting on a branch, it isn’t integrated. One common case when it seems obvious to use branches in version control is when making a large-scale change to your application. However there is an alternative to using branches: a technique called branch by abstraction.

Branch by abstraction: a pattern for making large-scale changes to your application incrementally on mainline.

The example Paul Hammant provides in his original blog entry on this technique is moving from Hibernate to iBatis. As it so happens, Go, the continuous integration and agile release management platform I work on, is presently moving from iBatis to Hibernate, and has been doing so for over a year now. We are also slowly moving our UI over from Velocity and JsTemplate to JRuby on Rails.

Both of these changes are being done slowly and incrementally, at the same time as developing new features, while checking in to mainline on our Mercurial repository multiple times a day. How do we do it?

Moving from iBatis to Hibernate

The team decided to move from iBatis to Hibernate for two reasons: first, we were able to use its ORM efficiently since we had control of our database schema, which saved us writing lots of custom SQL, and second, because its second-level cache helped performance.

Of course we didn’t want to move the whole codebase over at once. So as we started adding new functionality that required new calls to the database, we added these new calls using Hibernate, moving over old calls that used iBatis as required.

It is relatively straightforward to update the persistence logic incrementally because the Go codebase uses a standard layered architecture, with controllers using services that in turn use repositories. Because all the database access code is encapsulated in repository classes using the repository pattern, it’s a simple case of incrementally changing one repository class at a time from using iBatis to Hibernate. The service layer has no idea of the underlying persistence framework.

Moving from Velocity and JsTemplate to JRuby on Rails

We also wanted to move over from a Java-based UI stack to a JRuby on Rails stack, both because it was much easier to write tests for this stack, and because it speeded up UI development. Again, this change was made incrementally. When we created a new page in the application, we would create it using the JRuby on Rails stack, linking to the new page from the rest of the application once it was ready.

We would also move pages over whenever we wanted to make substantial changes to them. Again, the new version of the page would be developed using the new stack, and then we’d switch URIs in the rest of the application to point to the new version of the page once it was ready. At this point, we would remove the old version of the page. So while most of the UI in Go is now implemented using JRuby on Rails, there are still a couple of pages that use the old Java stack.

However you’d never know by looking at the pages, because they have identical styling. You have to look at the URI. Any URI that starts /go/tab is routing through the old Velocity stack. All the other URIs are routed via Rack to JRuby on Rails, which in turn calls through to the same Java service layer that the old UI also uses.

How branch by abstraction works

Branch by abstraction involves making large-scale changes to your system incrementally as follows:

1. Create an abstraction over the part of the system that you need to change.
2. Refactor the rest of the system to use the abstraction layer.
3. Create new classes in your new implementation, and have your abstraction layer delegate to the old or the new classes as required.
4. Remove the old implementation.
5. Rinse and repeat the previous two steps, shipping your system in the meantime if desired.
6. Once the old implementation has been completely replaced, you can remove the abstraction layer if you like.

In the iBatis/Hibernate example, the abstraction layer is the repository layer, which hides the implementation details of which persistence framework is being used. In the JRuby on Rails example, the abstraction layer is the servlet engine, which can dispatch either to the JRuby on Rails framework (using Rack) or to standard Java servlets by matching on the URI.

While Go is a relatively small project – it has less than ten developers, and it’s only been going for a few years – the same principles apply on projects of all sizes, and teams in ThoughtWorks have used this pattern successfully even on large and distributed projects.

Admittedly, branch by abstraction can add more overhead to the development process, especially in the case that the codebase is poorly structured. You need to think hard and move a bit slower in order to make changes incrementally in this fashion. But in many cases the upside is worth the extra effort, and the larger the restructuring, the more important it is to consider using branch by abstraction.

The key benefit of branch by abstraction is that your code is working at all times throughout the re-structuring, enabling continuous delivery. That means your release schedule is completely decoupled from your architectural changes, and thus you can stop working on the restructuring at any point to do something else that is higher priority, such as putting out a release with an exciting new feature you just thought up.

Branch by abstraction compared with branching in version control

Branch by abstraction is a somewhat misleading name, because of course it represents an alternative to using branching in version control when making large-scale changes to your system. Teams often use version control branches to make large-scale changes so that they can continue to develop functionality and fix bugs on mainline. The problem of course is that the merge back to mainline is guaranteed to be painful, and the amount of pain is a function both of how big the change you want to make is, and the amount of work you do on mainline in the meantime¹.

That means that the stronger the forces are that push you towards using a version control branch, the more painful it is going to be at the end when you have to merge. If you’re also using branches for features, the situation is made even worse. In general, using branches for features or large-scale changes is a bad idea, unless you are working in a small, experienced team, and the branches are very short lived (less than one day). Martin Fowler has excellent articles on why feature branching is bad, and how to use feature toggles as an alternative.

That doesn’t mean that all branching in version control is bad. It’s OK to branch in order to spike out an idea that you’re going to throw away. It’s also OK to branch upon releasing, provided you only use the release branch for small, critical bug-fixes. However teams who are practicing continuous deployment usually don’t bother with this, since it’s typically easier to fix any problems on mainline and roll forward than it is to roll back, because the delta between releases is so small.

The only other time it might be permissible to use branching is if your codebase uses the big ball of mud pattern. In this scenario even creating an abstraction layer can be hard. In order to do this, you must first find a “seam” (typically in the form of a set of interfaces if you’re using a statically typed OO language) that you can put the abstraction layer over. If a seam was not readily available, you would normally create one through a series of refactorings, but if that’s not possible for some reason then you might have to resort to creating a branch to get into a position to do this. Of course, this is an extreme move.

Relationship to other patterns

Relationship to refactoring. Refactoring has been defined as “a change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behavior”. In this sense, both of the examples given above of branch by abstraction are also examples of refactorings. Crucially though branch by abstraction is effectively a programme of related refactorings, which taken together result in a large-scale change to the architecture of the application.

Relationship to feature toggles. People often confuse branch by abstraction with feature toggles. Both are patterns that allow you to make changes to your system incrementally on mainline. The difference is that feature toggles are intended to allow the development of new features, while keeping those features invisible to users when the system is running. Feature toggles are thus used at deploy time or run time to choose whether a particular feature or set of features is visible in the application.

Branch by abstraction is a pattern for making large-scale changes to an application incrementally, and is thus a development technique. Branch by abstraction can of course be combined with feature toggles, such that – for example – you could switch between the iBatis and Hibernate implementations of a particular set of data access calls to compare the performance of them at runtime. But typically, the choice of implementation is chosen by the developers and either hard-coded or baked in at build time, perhaps through your dependency injection configuration.

Relationship to strangler application. The strangler application pattern involves incrementally replacing a whole system (usually legacy) with a completely new one. Thus it operates at a higher level of abstraction than branch by abstraction, which is for incrementally changing the implementation of a component of your system. The lines between the two start to blur if you have a service-oriented architecture.

Isn’t this just good object-oriented design? Yes. Code which follows the SOLID principles makes it very easy to apply this pattern, in virtue in particular of following the dependency inversion principle and the interface segregation principle (ISP). The ISP is important because it provides a nice level of granularity at which to switch out implementations. As my colleague David Rice points out, branch by abstraction is the only sensible way to change the implementation of some particular component. Martin Fowler makes the same point by sometimes defining a component as some part of a system that can be swapped out for another implementation.

---

¹ Another argument that is sometimes put forward is that distributed version control systems make merging so easy we shouldn’t be afraid of branching. This is misleading for two reasons. Firstly, as Martin Fowler points out, automated merge tools are incapable of catching semantic conflicts. Second, the longer the branch exists, the harder it is to merge, even with the best tools in the world. You don’t have to look too far on GitHub to find projects with forks that everyone would like to see merged, but have diverged so far from mainline that merging them would require substantial amounts of integration work.

Jailer is a database subsetting and sampling tool. It is a tool for data exporting, schema browsing, and rendering. It exports consistent, referentially intact row-sets from relational databases. It removes obsolete data without violating integrity. It is DBMS agnostic (by using JDBC), platform independent, and generates DbUnit datasets, hierarchically structured XML, and topologically sorted SQL-DML. Changes: [...]

Jailer is a database subsetting and sampling tool. It is a tool for data exporting, schema browsing, and rendering. It exports consistent, referentially intact row-sets from relational databases. It removes obsolete data without violating integrity. It is DBMS agnostic (by using JDBC), platform independent, and generates DbUnit datasets, hierarchically structured XML, and topologically sorted SQL-DML.

Changes: A bug regarding Outer Join for selecting dependent entries and Oracle 10 was fixed.

Tags: Database, Front-Ends, Software Development, Testing, Utilities, Archiving, Test Data, Database Subsetting

Licenses: Apache 2.0

Microsoft Office 365 Deployment Readiness Tool beta

http://community.office365.com/en-us/f/183/p/2285/8155.aspx#8155

 

Overview

---

The Office 365 Deployment Readiness Tool provides analysis of your on-premises environment in preparation for an Office 365 enterprise deployment. The readiness tool is integrated with the guidance provided in the web edition of the Microsoft Office 365 Beta Deployment Guide. The following assessments are provided:

• Domains
  • Email domain discovery and number of users leveraging each domain
• User Identity and Account Provisioning
  • Statistical information
  • Active Directory schema and forest/domain functional data
  • Trusts extract (checks for multi-forest constraints)
  • Directory Synchronization
    • Pre-requisite checks
    • Attribute assessment
  • Single sign on
    • Attribute assessment
• Exchange Online
  • Statistical information
  • Public folder, public delegates, and proxyaddresses extract
  • 3^rd party and unified messaging proxyaddresses information
• Lync Online
  • Statistical information
• SharePoint Online
  • User object count assessment
• Client and End User Experience
  • Summary of domain joined machines for rich experience/SSO readiness
• Network
  • Port analysis on certain Office 365 endpoints
  • DNS records assessment

''Non è stata fatta nemmeno la gara d'appalto per i due termovalorizzatori''
Rifiuti, Berlusconi: ''Da lunedì militari di nuovo a Napoli''


(05/05/2011 - 15:01:54)

Oggi la decisione in Cdm
Governo, via libera al rimpasto Premier: ci saranno altre nomine


(05/05/2011 - 15:48:03)

This white paper describes the FILESTREAM feature of SQL Server 2008, which allows storage of and efficient access to BLOB data using a combination of SQL Server 2008 and the NTFS file system. It covers choices for BLOB storage, configuring Windows and SQL Server for using FILESTREAM data, considerations for combining FILESTREAM with other features, and implementation details such as partitioning and performance.

This white paper is targeted at architects, IT Pros, and DBAs tasked with evaluating or implementing FILESTREAM. It assumes the reader is familiar with Windows and SQL Server and has at least a rudimentary knowledge of database concepts such as transactions.

In today’s society, data is generated at incredible rates and often needs to be stored and accessed in a controlled and efficient way. There are various technologies to do this and the choice of technology often depends on the nature of the data being stored – structured, semistructured, or unstructured:

• Structured data is data that can be easily stored in a relational schema, such as that representing sales data for a company. This can be stored in a database with a table of information for products the company sells, another with information about customers, and another that details sales of products to customers. The data can be accessed and manipulated using a rich query language such as Transact-SQL.
• Semistructured data is data that conforms to a loose schema but does not lend itself well to being stored in a set of database tables, such as data where each data point could have radically different attributes. Semistructured data is often stored using the xml data type in the Microsoft® SQL Server® database software and accessed using an element-based query language such as XQuery.
• Unstructured data may have no schema at all (such as a piece of encrypted data) or may be a large amount of binary data (many MBs or even GBs) which may seem to have no schema but in reality has a very simple schema inherent to it, such as image files, streaming video, or sound clips. Binary data in this case means data that can have any value, not just those that can be entered at a keyboard. These data values are commonly known as Binary Large Objects, or more simply BLOBs.

This white paper describes the FILESTREAM feature of SQL Server 2008, which allows storage of and efficient access to BLOB data using a combination of SQL Server 2008 and the NTFS file system. It covers the FILESTREAM feature itself, choices for BLOB storage, configuring the Windows® operating system and SQL Server for using FILESTREAM data, considerations for combining FILESTREAM with other features, and implementation details such as partitioning and performance.

http://msdn.microsoft.com/en-us/library/cc949109(v=sql.100).aspx 

Why We Chose MongoDB for Guardian.co.uk (SlideShare) -- they're using MongoDB's flexible schema, as schema upgrades were pain in their previous system (Oracle). I think of these as the database equivalent of dynamic typing in languages like Perl and Ruby. (via Paul Rowe) Solving Problems with Visual Analytics -- This book is the result of a community effort of...

[Prior Post in Series] [Next Post in Series]

In this part, I will deal with the fact that many remaining tables do not have clustered indexes on them. In my humble opinion, ANY reasonable order of the disk will perform better than a random order, so putting a clustered index on every table is a must-do.

The pattern is simple, if a table does not have a clustered index, then put a clustered index on the first column.  The typical behavior of database designers is to put the key indexes at the start of each table, so we are assuming this is true in this solution.

To obtain the list of tables lacking indexes, use this code snippet.

 

You may wish to manually add some indexes (based on your domain knowledge) before proceeding further.

Code Solution

The solution below grabs the first column only. A more advanced approach would be to walk each table and build the index from the leading columns that are integer values.

Summary

In this post we looked at loose-ends: tables lacking any clustered indexes.  We solved this problem by adding a clustered index using the first column.  A lot more analysis could be done on this database, but I ran out of time (after an apparent doubling of performance).

• Location: San Diego
• Compensation: Apply for more info
• Principals only. Recruiters, please don't contact this job poster.
• Please, no phone calls about this job!
• Please do not contact job poster about other services, products or commercial interests.

DBC ensuring your applications don’t fail due to a missing database structure object DBC (Database Compare) is a fast, comprehensive database structure and data comparison tool. DBC can compares the schema of two Microsoft SQL Server databases and quickly locates … Continue reading →

Windows Server 2003 R2 extended the Windows Server 2003 schema from schema version 30 to 31.  The update from 30 to 31 was schema file sch31.ldf which included support for DFS Replication (DFSr).  An upgrade from Windows Server 2003 to Windows Server 2008 schema transitions the schema to schema version 44.  This includes sch32.ldf through sch44.ldf. 

If an upgrade is performed from a domain that is currently schema version 30, the Windows Server 2008 ADPREP /forestprep command will include sch31.ldf in the schema update process.

Note: Windows Server 2008 R2 transitions the Active Directory Schema to schema version 47.

The updates to the Active Directory schema are:

Sch32.ldf

This adds new attributes of:

• msDS-KrbTgtLink: Used with RODCs to define which krbtgt_XXXX account corresponds to each RODC
• msDS-RevealedUsers: Used with RODCs to identify the user objects whose secrets have been disclosed to that RODC
• msDS-RevealedList: Identifies security principals whose current computer account passwords have been replicated to the RODC
• msDS-hasFullReplicaNCs: Identifies the partitions held as full replicas
• msDS-NeverRevealGroup: Used with RODCs to define which users, computers, and groups are not allowed to have their passwords cached on a RODC
• msDS-RevealOnDemandGroup: Used with RODCs to define which users, computers, and groups are allowed to have their passwords cached on a RODC
• msDS-SecondaryKrbTgtNumber: Identifies the protocol identification number associated with the secondary domain
• msDS-RevealedDSAs: Backlink For ms-DS-Revealed-Users and identifies which RODC holds that user's secret
• msDS-KrbTgtLinkBl: Backlink for the KrbTgtLink attribute
• msDS-IsDomainFor: Backlink for ms-DS-Has-Domain-NCs and identifies which DCs hold that partition as their primary domain
• msDS-IsFullReplicaFor: Backlink for ms-Ds-Has-Full-Replica-NCs and identifies which DCs hold that partition as a full replica
• msDS-IsPartialReplicaFor: Backlink for has-Partial-Replica-NCs and identifies which DCs hold that partition as a partial replica

After the addition of these attributes, Sch32.ldf then modifies the systemMayContain values of certain objects that may contain any or all of these new attributes.

The schema version is then increased to schema version 32.

 

Sch33.ldf

This adds new attributes of:

• msDS-isGC: Identifies the state of the Global Catalog on the DC
• msDS-isRODC: Shows whether a DC is a RODC
• msDS-SiteName: Lists the site name that corresponds to the DC
• msDS-AuthenticatedAtDC: Forwardlink for ms-DS-AuthenticatedTo-Accountlist and identifies which DC a user has authenticated to
• msDS-PromotionSettings: For a Computer, contains a XML string to be used for delegated DSA promotion
• msDS-SupportedEncryptionTypes: The encryption algorithms supported by user, computer or trust accounts
  • Note: The KDC uses this information while generating a service ticket for this account. Services/Computers may automatically update this attribute on their respective accounts in Active Directory, and therefore need write access to this attribute.
  • msDS-AuthenticatedToAccountlist: Backlink for ms-DS-AuthenticatedAt-DC and identifies which users have authenticated to this Computer

After the addition of these attributes, Sch33.ldf then modifies the msDS-Never-Reveal-Group and the msDS-Reveal-OnDemand-Group attributes and marks them as multi-valued.  It then modifies the systemMayContain values of certain objects that may contain any or all of these new attributes.

The schema version is then increased to schema version 33.

 

Sch34.ldf

Sch34.ldf adds the following attributes to the Schema:

• msDFSR-ReadOnly: Specifies whether the content is read-only or read-write
• msDFSR-Priority: Priority level
• msDS-AzObjectGuid: The unique and portable identifier of AzMan objects
• msDS-AzGenericData: AzMan specific generic data
• msDFSR-CachePolicy: On-demand cache policy options
• msDFSR-DeletedPath: Full path of the Deleted directory
• msFVE-RecoveryGuid: Contains the GUID associated with a Full Volume Encryption (FVE) recovery password
• msDS-SeniorityIndex: Contains the seniority index as applied by the organization where the person works
• msTPM-OwnerInformation: This attribute contains the owner information of a particular TPM
• msPKIDPAPIMasterKeys: Storage of encrypted DPAPI Master Keys for user
• msDS-PhoneticLastName: Contains the phonetic last name of the person
• msPKIRoamingTimeStamp: Time stamp for last change to roaming tokens
• msDFSR-DeletedSizeInMb: Size of the Deleted directory in MB
• msDS-PhoneticFirstName: Contains the phonetic given name or first name of the person
• msFVE-RecoveryPassword: Contains the password required to recover a Full Volume Encryption (FVE) volume
• msDS-PhoneticDepartment: Contains the phonetic department name where the person works
• msPKIAccountCredentials: Storage of encrypted user credential token blobs for roaming
• msRADIUS-FramedIpv6Route: Provides routing information to be configured for the user on the NAS
• msDS-PhoneticDisplayName: The phonetic display name of an object. In the absence of a phonetic display name the existing display name is used
• msDS-PhoneticCompanyName: Contains the phonetic company name where the person works
• ms-net-ieee-8023-GP-PolicyData: Contains all of the settings and data which comprise a Group Policy configuration for 802.3 wired networks
• ms-net-ieee-8023-GP-PolicyGUID: Contains a GUID which identifies a specific 802.3 Group Policy object on the domain
• msDFSR-MaxAgeInCacheInMin: Maximum time in minutes to keep files in full form
• ms-net-ieee-80211-GP-PolicyData: Contains all of the settings and data which comprise a Group Policy configuration for 802.11 wireless networks
• msRADIUS-FramedIpv6Prefix: Indicates an IPv6 prefix (and corresponding route) to be configured for the user
• ms-net-ieee-80211-GP-PolicyGUID: Contains a GUID which identifies a specific 802.11 Group Policy object on the domain
• msRADIUS-FramedInterfaceId: Indicates the IPv6 interface identifier to be configured for the user
• msDS-NC-RO-Replica-Locations: A linked attribute on a cross ref object for a partition and lists the DC which should host the partition in a readonly manner
• msDS-NC-RO-Replica-Locations-BL: Backlink attribute for ms-DS-NC-RO-Replica-Locations
• msDFSR-MinDurationCacheInMin: Minimum time in minutes before truncating files
• ms-net-ieee-8023-GP-PolicyReserved: Reserved for future use
• msRADIUS-SavedFramedIpv6Route: Provides routing information to be configured for the user on the NAS
• ms-net-ieee-80211-GP-PolicyReserved: Reserved for future use
• msRADIUS-SavedFramedIpv6Prefix: Indicates an IPv6 prefix (and corresponding route) to be configured for the user
• msRADIUS-SavedFramedInterfaceId: Indicates the IPv6 interface identifier to be configured for the user
• samDomainUpdates: Contains a bitmask of performed SAM operations on active directory

Sch34.ldf then adds the following classes to the Active Directory Schema:

• ms-net-ieee-8023-GroupPolicy: This class represents an 802.3 wired network Group Policy object. This class contains identifiers and configuration data relevant to an 802.3 wired network
• ms-net-ieee-80211-GroupPolicy: This class represents an 802.11 wireless network Group Policy object. This class contains identifiers and configuration data relevant to an 802.11 wireless network
• msFVE-RecoveryInformation: This class contains a Full Volume Encryption recovery password with its associated GUID
• nTDSDSARO: A subclass of Directory Service Agent which is distinguished by its reduced privilege level

After the addition of these attributes and classes, Sch34.ldf then modifies the systemMayContain values of certain objects that may contain any or all of these new attributes.

The schema version is then increased to schema version 34.

 

Sch35.ldf

Sch35.ldf adds the following attributes to the Schema:

• msDS-LastSuccessfulInteractiveLogonTime: The time that the correct password was presented during a Ctrl+Alt+Delete logon
• msDS-FailedInteractiveLogonCountAtLastSuccessfulLogon: The total number of failed interactive logons up until the last successful Ctrl+Alt+Delete logon
• msDS-FailedInteractiveLogonCount: The total number of failed interactive logons since this feature was turned on
• msDS-LastFailedInteractiveLogonTime: The time that an incorrect password was presented during a Ctrl+Alt+Delete logon

After the addition of these attributes, Sch35.ldf then modifies the systemMayContain values of the object CN=User,CN=Schema,CN=Configuration,DC=X (where DC=x is the Distinguished Name of the forest root domain) to include these new attributes.

The schema version is then increased to schema version 35.

 

Sch36.ldf

Then Sch36.ldf makes the following addition to the Schema:

• msDS-RevealedListBL: Backlink attribute for ms-DS-Revealed-List

After the addition of this one attribute, Sch36.ldf modifies the Search Flags value of the following attributes:

• CN=From-Server,CN=Schema,CN=Configuration,DC=X
• CN=msNPAllowDialin,CN=Schema,CN=Configuration,DC=X
• CN=msNPCallingStationID,CN=Schema,CN=Configuration,DC=X
• CN=msNPSavedCallingStationID,CN=Schema,CN=Configuration,DC=X
• CN=msRADIUSCallbackNumber,CN=Schema,CN=Configuration,DC=X
• CN=msRADIUSFramedIPAddress,CN=Schema,CN=Configuration,DC=X
• CN=msRADIUSFramedRoute,CN=Schema,CN=Configuration,DC=X
• CN=msRADIUSServiceType,CN=Schema,CN=Configuration,DC=X
• CN=msRASSavedCallbackNumber,CN=Schema,CN=Configuration,DC=X
• CN=msRASSavedFramedIPAddress,CN=Schema,CN=Configuration,DC=X
• CN=msRASSavedFramedRoute,CN=Schema,CN=Configuration,DC=X
• CN=ms-RADIUS-FramedInterfaceId,CN=Schema,CN=Configuration,DC=X
• CN=ms-RADIUS-SavedFramedInterfaceId,CN=Schema,CN=Configuration,DC=X
• CN=ms-RADIUS-FramedIpv6Prefix,CN=Schema,CN=Configuration,DC=X
• CN=ms-RADIUS-SavedFramedIpv6Prefix,CN=Schema,CN=Configuration,DC=X
• CN=ms-RADIUS-FramedIpv6Route,CN=Schema,CN=Configuration,DC=X
• CN=ms-RADIUS-SavedFramedIpv6Route,CN=Schema,CN=Configuration,DC=X
• CN=ms-FVE-RecoveryPassword,CN=Schema,CN=Configuration,DC=X
• CN=ms-FVE-RecoveryGuid,CN=Schema,CN=Configuration,DC=X
• CN=ms-TPM-OwnerInformation,CN=Schema,CN=Configuration,DC=X

The schema version is then increased to schema version 36.

 

Sch37.ldf

The LDF file Sch37.ldf modifies adds the following attributes to the schema:

• msDS-UserPasswordExpiryTimeComputed: Contains the expiry time for the user's current password
• msDS-PrincipalName: Account name for the security principal (constructed)
• msDFSR-OnDemandExclusionDirectoryFilter: Filter string applied to on demand replication directories
• msDFSR-DefaultCompressionExclusionFilter: Filter string containing extensions of file types not to be compressed
• msTSHomeDrive: Terminal Services Home Drive specifies a Home drive for the user
• msTSProperty01: Placeholder Terminal Server Property 01
• msTSProperty02: Placeholder Terminal Server Property 02
• msTSAllowLogon: Specifies whether the user is allowed to log on to the Terminal Server. The value is 1 if logon is allowed, and 0 if logon is not allowed.
• msTSExpireDate: TS Expiration Date
• msTSManagingLS: TS Managing License Server
• msDFSR-Options2: Object Options2
• msTSProfilePath: Terminal Services Profile Path specifies a roaming or mandatory profile path to use when the user logs on to the Terminal Server. The profile path is in the following network path format: \servername\profiles folder name\username
• msTSMaxIdleTime: Terminal Services Session Maximum Idle Time is maximum amount of time, in minutes, that the Terminal Services session can remain idle
• msTSHomeDirectory: Terminal Services Home Directory specifies the Home directory for the user
• msTSRemoteControl: Terminal Services Remote Control specifies the whether to allow remote observation or remote control of the user's Terminal Services session
• msTSWorkDirectory: Terminal Services Session Work Directory specifies the working directory path for the user
• msTSInitialProgram: Terminal Services Session Initial Program specifies the Path and file name of the application that the user wants to start automatically when the user logs on to the Terminal Server
• msTSLicenseVersion: TS License Version
• msTSMaxConnectionTime: Terminal Services Session maximum Connection Time is Maximum duration, in minutes, of the Terminal Services session
• msTSReconnectionAction: Terminal Services Session Reconnection Action specifies whether to allow reconnection to a disconnected Terminal Services session from any client computer
• msTSConnectClientDrives: Terminal Services Session Connect Client Drives At Logon specifies whether to reconnect to mapped client drives at logon
• msDFSR-CommonStagingPath: Full path of the common staging directory
• msTSMaxDisconnectionTime: Terminal Services Session Maximum Disconnection Time is maximum amount of time, in minutes, that a disconnected Terminal Services session remains active on the Terminal Server
• msTSDefaultToMainPrinter: Terminal Services Default To Main Printer specifies whether to print automatically to the client's default printer
• msTSConnectPrinterDrives: Terminal Services Session Connect Printer Drives At Logon specifies whether to reconnect to mapped client printers at logon
• msTSBrokenConnectionAction: Terminal Services Session Broken Connection Action specifies the action to take when a Terminal Services session limit is reached
• msDFSR-DisablePacketPrivacy: Disable packet privacy on a connection
• msDFSR-CommonStagingSizeInMb: Size of the common staging directory in MB
• msDFSR-OnDemandExclusionFileFilter: Filter string applied to on demand replication files
• msDFSR-StagingCleanupTriggerInPercent: Staging cleanup trigger in percent of free disk space

 

After these attributes have been added, Sch37.ldf modifies security on Terminal Services objects and then updates the mayContain values of Terminal Services and DFSr objects.

The schema version is then increased to schema version 37.

 

Sch38.ldf

Sch38.ldf only makes one change.  This change is to the CN=ms-DS-AuthenticatedAt-DC,CN=Schema,CN=Configuration,DC=X attribute.  The modification that is made to this attribute is to mark this attribute as systemOnly.

After this change, the schema version is still increased to version 38.

 

Sch39.ldf

Sch39.ldf begins by modifying the following attributes:

• msFVE-KeyPackage: Contains a volume's BitLocker encryption key secured by the corresponding recovery password
• msFVE-VolumeGuid: Contains the GUID associated with a BitLocker-supported disk volume
• msDS-HABSeniorityIndex: Contains the seniority index as applied by the organization where the person works

Sch39.ldf then modifies the adminDescription, searchFlags, and rangeUppper of the attributes:

• CN=ms-FVE-RecoveryPassword,CN=Schema,CN=Configuration,DC=X
• CN=ms-FVE-RecoveryGuid,CN=Schema,CN=Configuration,DC=X
• CN=ms-TPM-OwnerInformation,CN=Schema,CN=Configuration,DC=X
• CN=ms-FVE-RecoveryInformation,CN=Schema,CN=Configuration,DC=X
• CN=msSFU-30-Posix-Member,CN=Schema,CN=Configuration,DC=X

Lastly, Sch39.ldf also updates the systemMayContain and mayContain values of additional objects in the Schema which could contain any of these attributes.

The schema version is then increased to schema version 39.

 

Sch40.ldf

The LDF file Sch40.ldf adds many attributes to the schema.  Half of these attributes are used with Fine Grained Password policies and the other half are used with Terminal Server Licensing.  This list is all of the attributes that are added to the Active Directory schema:

• msDS-PasswordReversibleEncryptionEnabled: Password reversible encryption status for user accounts
• msDS-NcType: A bit field that maintains information about aspects of a NC replica that are relevant to replication
• msDS-PSOAppliesTo:  Links to objects that this password settings object applies to
• msDS-PSOApplied: Password settings object applied to this object
• msDS-ResultantPSO: Resultant password settings object applied to this object
• msDS-LockoutDuration: Lockout duration for locked out user accounts
• msDS-LockoutThreshold: Lockout threshold for lockout of user accounts
• msDS-MinimumPasswordAge: Minimum Password Age for user accounts
• msDS-MaximumPasswordAge: Maximum Password Age for user accounts
• msDS-MinimumPasswordLength: Minimum Password Length for user accounts
• msDS-PasswordHistoryLength: Password History Length for user accounts
• msDS-LockoutObservationWindow: Observation Window for lockout of user accounts
• msDS-PasswordComplexityEnabled: Password complexity status for user accounts
• msDS-PasswordSettingsPrecedence: Password Settings Precedence
• msTSManagingLS2: Issuer name of the second TS per user CAL
• msTSManagingLS3: Issuer name of the third TS per user CAL
• msTSManagingLS4: Issuer name of the fourth TS per user CAL
• msTSExpireDate2: Expiration date of the second TS per user CAL
• msTSExpireDate3: Expiration date of the third TS per user CAL
• msTSExpireDate4: Expiration date of the fourth TS per user CAL
• msTSLSProperty01: Placeholder Terminal Server License Server Property 01
• msTSLSProperty02: Placeholder Terminal Server License Server Property 02
• msTSLicenseVersion2: Version of the second TS per user CAL
• msTSLicenseVersion3: Version of the third TS per user CAL
• msTSLicenseVersion4: Version of the fourth TS per user CAL
• msDS-IsUserCachableAtRodc: For a Read-Only Domain Controller (RODC), Identifies whether the specified user's secrets are cacheable

It is important to note that these password policy related attributes (i.e. msDS-LockoutDuration, msDS-LockoutThreshold, msDS-MinimumPasswordAge, etc.) are not simply an update to the existing Lockout Duration, Lockout Threshold, Minimum Password Age, etc. settings that administrators are used to see in a Password Policy.  Password Policy settings on a Windows 2000 and Windows Server 2003 domain controller are contained in the registry of the domain controller, not as attributes held within Active Directory.

Windows Server 2008 in Domain Functional Level 2008 allows for multiple Fine Grained Password Policies per domain.  For this to exist, attributes that correspond to each setting needed to be introduced to the schema.

Sch40.ldf then modifies attributes that have been created during the schema update and also updates various objects in the schema.  These modifications are searchFlags, mayContain, systemMayContain, and possPosition values.

After these changes, sch40.ldf then creates two new classes that pertain to Fine Grained Password Policies.  These classes are:

• msDS-PasswordSettingsContainer: Container for password settings objects
• msDS-PasswordSettings: Password settings object for accounts
  • Note: This classSchema object is created with a list of systemMustContain OIDs.  This is a list of attributes that a Password Settings Object (PSO) must contain or else the creation of the PSO will fail.

After the classes are created, the sch40.ldf file then modifies more systemMayContain values of other objects.

The schema version is then increased to schema version 40.

 

Sch41.ldf

Sch41.ldf makes only a few changes to objects that exist in the schema.  First, modifications are made to the systemMayContain values of the objects:

• CN=NTDS-DSA,CN=Schema,CN=Configuration,DC=X
• CN=NTDS-DSA,CN=Schema,CN=Configuration,DC=X
• CN=NTDS-DSA,CN=Schema,CN=Configuration,DC=X
• CN=ms-DS-PSO-Applied,CN=Schema,CN=Configuration,DC=X
• CN=ms-DS-Resultant-PSO,CN=Schema,CN=Configuration,DC=X

Second, modifications are made to the rightsGUID values of the objects:

• CN=MS-TS-GatewayAccess,CN=Extended-Rights,CN=Configuration,DC=X
• CN=Terminal-Server-License-Server,CN=Extended-Rights,CN=Configuration,DC=X

The schema version is then increased to schema version 41.

 

Sch42.ldf

Schema update 42 makes modifications to over 360 objects in the schema.  The modifications that are made to these objects are adding the schemaFlagsEx attribute to each object and setting a value of 1.  The code for each modification looks like this:

changetype: ntdsSchemaModify

add: schemaFlagsEx

schemaFlagsEx: 1

Only the operating system can modify the SchemaFlagsEX value and this value specifies whether an attribute can be part of the filtered attribute set.

After the update to all 360+ attributes, the schema version is increased to schema version 42.

 

Sch43.ldf

Sch43.ldf adds the following attributes to the Active Directory Schema:

• msDFS-SchemaMajorVersion: Major version of schema of DFS metadata
• msDFS-SchemaMinorVersion: Minor version of schema of DFS metadata
• msDFS-GenerationGUIDv2: To be updated each time the entry containing this attribute is modified
• msDFS-NamespaceIdentityGUIDv2: To be set only when the namespace is created. Stable across rename/move as long as namespace is not replaced by another namespace having same name
• msDFS-LastModifiedv2: To be updated on each write to the entry containing the attribute
• msDFS-Ttlv2: TTL associated with DFS root/link. For use at DFS referral time
• msDFS-Commentv2: Comment associated with DFS root/link
• msDFS-Propertiesv2: Properties associated with DFS root/link
• msDFS-TargetListv2: Targets corresponding to DFS root/link
• msDFS-LinkPathv2: DFS link path relative to the DFS root target share (i.e. without the server/domain and DFS namespace name components). Use forward slashes (/) instead of backslashes so that LDAP searches can be done without having to use escapes
• msDFS-LinkSecurityDescriptorv2: Security descriptor of the DFS links's reparse point on the filesystem
• msDFS-LinkIdentityGUIDv2: To be set only when the link is created. Stable across rename/move as long as link is not replaced by another link having same name
• msDFS-ShortNameLinkPathv2: Shortname DFS link path relative to the DFS root target share (i.e. without the server/domain and DFS namespace name components). Use forward slashes (/) instead of backslashes so that LDAP searches can be done without having to use escapes
• msDFS-NamespaceAnchor: DFS namespace anchor
• msDFS-Namespacev2: DFS namespace
• msDFS-Linkv2: DFS Link in DFS namespace
• msDFS-DeletedLinkv2: Deleted DFS Link in DFS namespace
• addressBookRoots2: Used by Exchange. Exchange configures trees of address book containers to show up in the MAPI address book. This attribute on the Exchange Config object lists the roots of the address book container trees
• globalAddressList2: This attribute is used on a Microsoft Exchange container to store the distinguished name of a newly created global address list (GAL)
• templateRoots2: This attribute is used on the Exchange config container to indicate where the template containers are stored. This information is used by the Active Directory MAPI provider

Once these attributes have been created the schema version is incremented to schema version 43.

 

Sch44.ldf

Schema Update 44 only does modifications to some objects that already exist.  These modifications are changing systemMayContain, showInAdvancedViewOnly, searchFlags, and adminDescription.  Once this is complete, the schema version is incremented to schema version 44.  At this point, the Windows Server 2008 ADPREP /forestprep is complete

Additional LDF files for Windows Server 2008 R2 ADPREP /forestprep

 

Sch45.ldf

Sch45.ldf adds the following attributes to the Active Directory schema:

• msDS-USNLastSyncSuccess: The USN at which the last successful replication synchronization occurred
• isRecycled: Is the object recycled (for use with AD Recycle Bin)
• msDS-OptionalFeatureGUID: GUID of an optional feature
• msDS-EnabledFeature: Enabled optional features
• msImaging-PSPString: Schema Attribute that contains the XML sequence for this PostScan Process
• msDS-OIDToGroupLink: For an OID, identifies the group object corresponding to the issuance policy represented by this OID
• msDS-OIDToGroupLinkBl: Backlink for ms-DS-OIDToGroup-Link; identifies the issuance policy, represented by an OID object, which is mapped to this group
• msImaging-PSPIdentifier: Schema Attribute that contains the unique identifier for this PostScan Process
• msDS-HostServiceAccount: Service Accounts configured to run on this computer
• msDS-HostServiceAccountBL: Service Accounts Back Link for linking machines associated with the service account
• msDS-RequiredDomainBehaviorVersion: Required domain function level for this feature
• msDS-RequiredForestBehaviorVersion: Required forest function level for this feature
• msPKI-CredentialRoamingTokens: Storage of encrypted user credential token blobs for roaming
• msDS-LocalEffectiveRecycleTime: Recycle time of the object in the local DIT
• msDS-LocalEffectiveDeletionTime: Deletion time of the object in the local DIT
• msDS-LastKnownRDN: Holds original RDN of a deleted object
• msDS-EnabledFeatureBL: Scopes where this optional feature is enabled
• msDS-DeletedObjectLifetime: Lifetime of a deleted object
• msDS-OptionalFeatureFlags: An integer value that contains flags that define behavior of an optional feature in Active Directory
• msPKI-Enrollment-Servers: Priority, authentication type, and URI of each certificate enrollment web service
• msPKI-Site-Name: Active Directory site to which the CA machine belongs
• msTSEndpointData: This attribute represents the VM Name for machine in TSV deployment.
• msTSEndpointType: This attribute defines if the machine is a physical machine or a virtual machine.
• msTSEndpointPlugin: This attribute represents the name of the plugin which handles the orchestration.
• msTSPrimaryDesktop: This attribute represents the forward link to user's primary desktop.
• msTSSecondaryDesktops: This attribute represents the array of forward links to user's secondary desktops.
• msTSPrimaryDesktopBL: This attribute represents the backward link to user.
• msTSSecondaryDesktopBL: This attribute represents the backward link to user.
• msImaging-PSPs: Container for all Enterprise Scan Post Scan Process objects.
• msDS-OptionalFeature: Configuration for an optional DS feature.
• msImaging-PostScanProcess: Enterprise Scan Post Scan Process object.
• msDS-ManagedServiceAccount: Service account class is used to create accounts that are used for running Windows services.

Then, modifications are made to various objects in the schema which contain values such as systemMayContain and appliesTo.  Lastly, the objects CN=Optional Features,CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=X and CN=Recycle Bin Feature,CN=Optional Features,CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=X are created as optional features that can be enabled with Windows Server 2008 R2.

When this is complete the schema version is incremented to 45.

 

Sch46.ldf

The LDF file Sch46.ldf makes one change to the object CN=ms-DS-Managed-Service-Account,CN=Schema,CN=Configuration,DC=X.  The change that is made is to modify the defaultHidingValue and configure it with a value of FALSE.

When this is complete the schema version is incremented to 46.

 

Sch47.ldf

Sch47.ldf modifies only two objects in the schema.  These objects are CN=NTDS-DSA,CN=Schema,CN=Configuration,DC=X and CN=ms-DS-Managed-Service-Account,CN=Schema,CN=Configuration,DC=X.  The modification to the first is a modification to the systemMayContain value.  The modification to the second object is a deletion of the systemPossSuperiors value.

When this is complete the schema version is incremented to schema version 47 and the ADPREP /forestprep for Windows Server 2008 R2 is complete.

 

 

 

William Cook is documenting a new graph based "programming model" on his blog. It seems quite early in development, probably too early to offer qualified comments, but the claimed properties are interesting:

Ensō is a theoretically sound and practical reformulation of the concepts of model-driven software development. Ensō is based on first-class structural descriptions, invertable transformations, generic operations and interpretation.

Structures in Ensō are a specialized kind of graph, whose nodes are either primitive data or collections of observable properties, whose values are either nodes or collections of nodes. (snip) The key point is that structures in Ensō are viewed holistically as graphs, not as individual values or traditional sums-and-products data structures.

A structural description, or schema, specifies some of the observable properties of structures. Schemas are used to check the consistency structures. Some properties can be checked while the structure is being created, but other can only be checked once the structure is complete. Ensō allows modification of structures, which is necessary to create cyclic graphs, but also allows valid structures to be sealed to prevent further changes.

Invertible transformations are used to map one structure into another kind of structure, such that mapping can be inverted to (partially) recover the original structure from the result. One common kind of transformation is called a grammar, which is an invertible transformation between structures and text. Text grammars are invertable because they can be used for parsing and also rendering

Excerpt: One of the most fundamental ways in which cultural beliefs, practices, and ideologies influence psychological processes is in the cognitive schema or self-construal style that people use to think about themselves and their relation to others. In particular, previous cultural psychology research has identified two main styles of self-construal: independent (or individualistic) and interdependent (or collectivistic). Individuals from independent...

West Division

Summary:
Responsible for coordinating the operation of web based applications within a service oriented architecture. Under minimal supervision performs aspects of Database Administration including, hardware and software evaluation and installation, logical and physical database design and implementation, structured testing of database designs, security administration, backup and recovery planning, performance and tuning, research and evaluation of new tools and techniques, and development of policies, standards and procedures. Participates with the implementation and support of development systems, QA systems, and production systems at data centers, including all network and systems-related infrastructure components. Has in-depth experience, knowledge and skills in own discipline. Usually determines own work priorities. Acts as a resource for colleagues with less experience.

Core Responsibilities:
-Coordinate integration of web based business applications within Oracles Service Oriented Architecture Suite.
- Communicates and understands impact of database changes to affected parties and makes recommendations, as appropriate. Maintains hardware platforms to perform at optimum response, capacity planning, and implementation of new databases due to conversions.
- Collaborate with application users, business process managers, database administrators, security administrators and stakeholders to define application requirements.
- Consults with Project Managers to design appropriate solutions. Assists the development teams with application schema designs and applies best practices.
- Commissions and installs new applications.
- Develops, revises, and implements policies, standards, procedure,s and guidelines necessary to effectively administer the database environments.
- Develops, manages, and tests backup and recovery plans.
- Designs/tunes, maintains and supports database systems for highly available, high-volume internet applications.
- Other duties and responsibilities as assigned.
- Consistent exercise of independent judgment and discretion in matters of significance.
- Regular, consistent and punctual attendance. Must be able to work nights and weekends, variable schedule(s) as necessary.


Required Skills:
Required Skills/Experience:
* Java, Java EE
* Java-based web services
* SOAP

Desired Skills/Experience:
* Oracle SOA Suite
* Adobe Flex
*Service Oriented Architecture (SOA)
* Python
* FME

Education Level: -Bachelors Degree or Equivalent
Field of Study: - Information Sciences/Technology, - Computer Science
Years Experience: - Generally requires 5-8 years related experience
Comcast is an EEO/AA/Drug Free Workplace.

• Lieu : Puyallup
• Intéressés seulement. Recruteurs, veuillez ne pas contacter ce demandeur d'emploi.
• Les appels téléphoniques au sujet de cette offre ne sont pas autorisés.
• Veuillez ne pas contacter l'annonceur de l'offre d'emploi pour d'autres services, produits ou intérêts commerciaux.

You've migrated from Exchange 2003 to Exchange 2010. Your users are reporting that they're unable to modify distribution groups that they could in the past. In How to Manage Groups that I already own in Exchange 2010?, we showed you that in Exchange 2010, users had permissions to create new distribution groups and remove the distribution groups they owned.

This did help a few of your users but some are still unable to modify their distribution groups. You need to look into this….

Issue

When User1 tries to add a new member to the DistributionGroup1 distribution group, she gets this error:

Changes to the distribution group list membership could not be saved. You do not have sufficient permissions to perform this operation on this object.

Figure 1: User gets a permissions error when trying ot modifying a distribution group membership in Microsoft Outlook

In the past, User1 was able to add/remove members to the distribution group by using Outlook and didn't need to call the help desk for assistance. What has changed?

You know that you had previously configured SecurityGroup1 to be able to manage this distribution group. Did someone make a change to User1’s security group membership? You look at ADUC first. No change – the user's still a member of SecurityGroup1.

Figure 2: The user's still a member of the security group that's configured to manage the distribution group

Next, you check the distribution group configuration in ADUC to verify that DistributionGroup1 is still being managed by SecurityGroup1.

Figure 3: The user's still a member of the security group that's configured to manage the distribution group

Okay, so what’s the deal here? You know you recently migrated to Exchange 2010. So you take a look at DistributionGroup1 in EMC (which reveals that it's managed by SecurityGroup1, but also displays an 'Object Not Found' error).

Figure 4: EMC displays the security group with an 'Object not found' error

Why is Exchange 2010 doing this?

This behavior is by design. In Exchange 2010, distribution groups can't be managed by groups - only individual users can manage groups. So it's possible that using Exchange 2003, you used groups to manage a distribution group. Group ownership was handled at a different level. Now that these mailboxes have been moved to Exchange 2010, members of these groups can't modify the group.

So are there any workarounds?

We've created a script to work around this limitation. Download Set-DistributionGroupOwners.ps1 from the Script Center.

The script will allow you to simulate a group having ownership of a distribution group in Exchange 2010. The script can be run in three different modes depending on the switches you pass.

1. Mode 1 – Set Ownership for a particular distribution group. Modifications to the ManagedBy attribute are not set at this time. It will simply modify a Custom Attribute to have the information needed later when the script sets the ManagedBy attribute.
2. Mode 2 – Modify the ManagedBy attribute of a specific distribution group so the members of either a security group or distribution group can manage it.
3. Mode 3 – Is designed to be run as a scheduled task and ensure individual members of a group have ownership of the Distribution Group which they are set to own. This mode is used if you prefer to automate the process and perhaps run it nightly to look for any changes to security group and distribution group membership.

How do I run this thing?

1. Windows 2008 R2 is required to run the script. This was needed in order to support Security Groups owning Distribution Groups. This does not have to be run on an Exchange server but the Exchange management tools must be installed.
2. Download the script. The Set-DistributionGroupOwners.ps1 script is now available from TechNet Script Center. Download it and change the file extension to .ps1. I recommend Set-DistributionGroupOwners.ps1.
3. Decide what CustomAttribute can be used in your environment. The script will populate the Distinguished Name (DN) of the group (specified in the ManagedBy attribute of the distribution group you want to manage) in a custom attribute. By default, this is CustomAttribute5, but this can easily be changed to use one of the fifteen custom attributes in the default schema. Find the following in Set-DistributionGroupOwners.ps1:

   $dn_storage = "CustomAttribute5"

   Change CustomAttribute5 to the custom attribute of your choice.

   You're now ready to run the script.

Mode 1 - Set Ownership of a Group

In this mode, run the script with both -DistributionGroup AND –GroupOwner parameters. Specify the distribution group (-DistributionGroup) and the group that you want to manage it (-GroupOwner). This will then set the DN of the owning group from –GroupOwner into the CustomAttribute you specified on the Distribution Group from –DistributionGroup.

In order to have DistributionGroup1 managed by SecurityGroup1, you would run the following:

Figure 5: The script populates the group owner's DN in a custom attribute

A dump of the DL above shows that CustomAttribute5 is populated with the DN of SecurityGroup1 and the ManagedBy attribute remains with only SecurityGroup1 listed. Mode 2 is needed in order for members of SecurityGroup1 to be able to modify DistributionGroup1.

Mode 2 - Modify the ManagedBy attribute for one Group

Neither Mode 2 nor Mode 3 will work until you have set the value of the customer attribute to the DN of the Owning Group. If you have already run the Script in Mode 1, then Mode 2 will configure the ManagedBy attribute for a single group. To run in Mode 2, simply specify only the –DistrubitionGroup switch and list the DL that you want to have processed.

In our example, we have specified group, DistributionGroup1. This step will then set members of the owning group on the ManagedBy attribute. They are now listed by individual name.

Figure 6: The script populates the distribution group's ManagedBy attribute with (individual) members of the owning group

Mode 3 – Run the Script as a Scheduled Task to look all new modifications to Group Ownership

When you run the script with no switches, it will search AD for all Groups that have the defined custom attribute set to a DN. It will then process all of them as in Mode 2.

The script is designed to be run in this mode as either a one off type operation when you know updates are needed or as a scheduled task to keep everything in sync. A key point is that when populating the ManagedBy attribute, it overwrites existing values with the current members of the owning group.

Note: This script in its current form will NOT chase nested groups. It will only take members of the group that's listed in the defined custom attribute. There are currently no plans to add nested group functionality to the script.

Many special thanks to our scripting genius Matt Byrd, whose motto continues to be “If this needs to be done more than once, it’s getting scripted!”

Tammy Anthony

by Stoney deGeyter

I learned a new word recently. Not a new word, really, but a new phrase combined from two of those really big words that normal people rarely ever use. That phrase is "Concatenation Schema".

WTH???

Yeah, that's what I thought. But, suffice it to say, it was a very simplified way of saying something that took me three paragraphs to explain. It's nice to be able to add a new phrase into your vernacular (another really big word for me) that nicely sums up something that is difficult to explain.

In the plainest, most simplest terms I could come up with, Concatenation Schema is just another way of saying "default content that changes dynamically based on category, sub-category, and product related information."

How Do You Build a Concatenation Schema?

The idea behind Concatenation Schema is to develop the SEO guidelines your programmers will use to populate key areas of your site pages. These key areas can include, but are not limited to:

• Title tag
• Meta description
• Headings
• ALT text
• Body content

The first step is to figure out what your variables are. It might be categories, sub-categories, filters, titles, product names, years, makes, models, brands, styles, colors, etc.

If your site's navigation works like: CATEGORY > SUB-CATEGORY > PRODUCT, then those are your three variables. If visitors navigate via BRAND > CATEGORY > STYLE > PRODUCT, then use those as your variables. You might have one or two more variables thrown in, depending on the type of products or services you do, but you want to try to keep it as simplified as possible.

Let's use an example of a site that sells auto parts and accessories. Here are some examples of three different title tag schemas you could use for three different category levels:

Title Tag Schema 1

[MAKE] [PART] | Replacement [PART] for [MAKE] [VEHICLE]s

Examples:

• Ford Headlamp | Replacement Headlamp for Ford SUVs
• 
  
• Dodge Floor Mats | Replacement Floormats for Dodge Trucks
• 
  
• Mazda Fan Belt | Replacement Fan Belt for Mazda Cars
• 
  

Title Tag Schema 2

[YEAR] [MAKE] [PART] | Replacement [PART] for [MAKE] [VEHICLE]

Examples:

• 
  
• 2010 Ford Headlamp | Replacement Headlamp for Ford SUVs
• 
  
• 1998 Dodge Floor Mats | Replacement Floor Mats for Dodge Trucks
• 
  
• 2011 Mazda Fan Belt | Replacement Fan Belt for Mazda Cars
• 
  

Title Tag Schema 3

Replacement [PART] for [YEAR] [MAKE] [MODEL]

Examples:

• 
  
• Replacement Headlamp for 2010 Ford Explorer
• 
  
• Replacement Floor Mats for 1998 Dodge Dakota
• 
  
• Replacement Fan Belt for 2011 Mazda 6
• 
  

Please note, I make no representation to the SEO value of the examples above, nor do I suggest this is the best way to format the title tags. I only mean to provide examples on how to create and use schemas.

After the title tag, we might move on to the meta description:

Meta Description Schema 1

Find replacement [MAKE] [PART]. We have everything you need for [MAKE] or other [VEHICLE] [PART]s. Free overnight delivery on orders over $300 at MyWebsite.com.

Examples:

• Find replacement Ford headlamps. We have everything you need for Ford or other SUV headlamps. Free overnight delivery on orders over $300 at MyWebsite.com.
• Find replacement Dodge floor mats. We have everything you need for Dodge or other truck floor mats. Free overnight delivery on orders over $300 at MyWebsite.com.
• Find replacement Mazda fan belt. We have everything you need for Mazda or other car headlamps. Free overnight delivery on orders over $300 at MyWebsite.com.

Meta Description Schema 2

Find [YEAR] [MAKE] [VEHICLE]s [PART]. Whether you're looking for a [MAKE] [PART] or for any other [VEHICLE]. Free overnight delivery on orders over $300 at MyWebsite.com.

Examples:

• 
  
• Find 2010 Ford SUVs headlamps. Whether you're looking for a Ford headlamp or for any other SUV. Free overnight delivery on orders over $300 at MyWebsite.com.
• Find 1998 Dodge trucks floor mats. Whether you're looking for Dodge floor mats or for any other truck. Free overnight delivery on orders over $300 at MyWebsite.com.
• Find 2011 Mazda cars fan belts. Whether you're looking for Mazda fan belts or for any other car. Free overnight delivery on orders over $300 at Mywebsite.com.
• 
  

Meta Description Schema 3

Find replacement [YEAR] [MAKE] [MODEL] [VEHICLE] [PART]. Whether you're looking for a [MAKE] [PART], or [PART] for any other [MAKE] [VEHICLE], you'll find what you need at MyWebsite.com.

Examples:

• 
  
• Find replacement 2010 Ford Explorer SUV headlamps. Whether you're looking for a Ford headlamp, or headlamps for any other Ford SUV, you'll find what you need at MyWebsite.com.
• Find replacement 1998 Dodge Dakota truck floor mats. Whether you're looking for Dodge floor mats, or floor mats for any other Dodge truck, you'll find what you need at MyWebsite.com.
• Find replacement 2011 Mazda 6 car fan belts. Whether you're looking for Mazda fan belts, or fan belts for any other Mazda car, you'll find what you need at MyWebsite.com.
• 
  

From here you can get the gist of how to continue with the Concatenation Schema for the rest of the important site elements that we noted above, plus any others that you may need.

You can also do the same with textual content; however, creating unique text is recommended for each page.

How Many Schemas Do You Need?

Most sites will require more than one set of Concatenation Schema. In the simplest terms, you would have a schema for each section of the site:

• Top category pages
• Sub category pages
• Product pages

The image below depicts a site with three categories, each with three sub-categories and nine products per sub-category. Each color represents a different schema needed.

In reality, however, it might be a bit more complex. You can't always create a perfect schema that will cover every category of your site. In some cases you might need to create a different schema for different categories. So it might actually play out something more like this:

We just went from three distinct Concatenation Schemas to a total of 11. But, before you get your panties all wadded up, by performing SEO this way, you've actually reduced the total number of pages you have to "optimize." Instead of optimizing 93 pages, you're only creating 11 schemas. That's a big reduction in time, even for a "small" site, such as the one depicted here.

Now imagine your own site, which has how many categories, sub-categories, and products? By building a few Concatenation Schemas, you're able to turn a large task into a (relatively) smaller one!

Be sure and visit our small business news site.

While data size is a consideration, it is not the only one, and I'd like to provide an operational consideration here if I may. Most of our customers get hooked on the flexibility well before PB+ scale out becomes a driver. The "cost" for experimentation and mixed data set analytics compared to conventional approaches is what really matters more than size when getting started. There is no schema to get "wrong", no ELT and MDM required before getting started (notice I didn't say they wouldn't be the need for those in the future however) so the "cost" to get started in low and the price for failure is equally low. Anyway, something to consider.

DbSchema is a database tool for managing database data and schema, it brings both database schema and data in a single unified workflow. This provides an efficient and productive environment to work in, with improved awareness of software schema architecture and easy access to software data. Support for offline work and schema synchronization, make DbSchema an excelent tool for team work as well as a outstanding tool for converting and merging database schemas.

This blog post complements and should be read in conjunction with the previous post Carving SQLite databases from unallocated clusters. In that post I looked at the information available within an SQLite database that may assist in carving one from unallocated clusters.

You will remember from the earlier blog post that SQLite databases are divided into equally sized pages and SQLite database files always consist of an exact number of pages. The page size for a database file is determined by the 2 byte integer located at an offset of 16 bytes from the beginning of the database file. The first page in an SQLite database is numbered page 1.

Auto-vacuum capable
Auto-vacuum capable SQLite databases make use of Pointer Map pages along with the other page types detailed in the earlier blog post. It is probably helpful to provide some information about what an auto-vacuum capable database is.

In a non-auto-vacuum-capable SQLite database when information is deleted the pages where it was stored are added to a list of free pages and these pages can be reused the next time data is inserted. Therefore, should data be deleted the file size of the database does not decrease. If a lot of data is deleted and it becomes necessary to shrink the database size the SQL VACUUM command can be run. This has the effect of reorganising the database from scratch and removing any free pages completely, thus making the database smaller.

When Auto-vacuum is enabled all free pages are moved to the end of the database file and the database file is truncated to remove the free pages at every transaction commit, thus removing free pages automatically.

Pointer Map Pages
The purpose of the Pointer Map is to facilitate moving pages from one position in the database file to another as part of auto vacuum. When a page is moved, the pointer in its parent must be updated to point to the new location. Pointer Maps are used to provide a lookup table to quickly determine what a pages parent page is. They only exist within auto-vacuum capable databases, which require the 32 bit integer value, read big endian, stored at byte offset 52 of the database header to be non-zero.

In auto-vacuum-capable SQLite databases page 2 of the database is always a Pointer Map page. Pointer Map pages store a 5 byte record relating to every page that follows the Pointer Map page. For example if we have an auto-vacuum-capable database that has 24 pages (each of 4096 bytes in size) in total, page 1 will contain the database header and the database schema and the next page, page 2, will be a Pointer Map page. This Pointer Map page will contain a 5 byte record for every one of the remaining 22 pages taking up 110 bytes of space within the page. The first 5 byte record begins at the very beginning of the Pointer Map page and therefore in a 4096 byte page a maximum of 819 (4096/5) records can be stored. If the database has more than 821 pages (when using a page size of 4096 bytes) page 822 would be an additional Pointer Map page that would contain records for the next 819 pages following this second Pointer Map page. Further additional Pointer Map pages can be added in the same way. Pointer Map pages do not store records relating to Pointer Map pages or page 1 of the database.

Pointer Map 5 byte records are structured with 1 byte indicating a Page Type and then 4 bytes, decoded big endian, referencing the parent page number as follows:

• 0x01 0x00 0x00 0x00 0x00
  This record relates to a B-tree root page which obviously does not have a parent page, hence the page number being indicated as zero.
  
• 0x02 0x00 0x00 0x00 0x00
  This record relates to a free page, which also does not have a parent page.
  
• 0x03 0xVV 0xVV 0xVV 0xVV (where VV indicates a variable)
  This record relates to the first page in an overflow chain. The parent page number is the number of the B-Tree page containing the B-Tree cell to which the overflow chain belongs.
  
• 0x04 0xVV 0xVV 0xVV 0xVV (where VV indicates a variable)
  This record relates to a page that is part of an overflow chain, but not the first page in that chain. The parent page number is the number of the previous page in the overflow chain linked-list.
  
• 0x05 0xVV 0xVV 0xVV 0xVV (where VV indicates a variable)
  This record relates to a page that is part of a table or index B-Tree structure, and is not an overflow page or root page. The parent page number is the number of the page containing the parent tree node in the B-Tree structure.
  
  

Figure 1

The screenshot at Figure 1 shows the Pointer Map page of an iPhone SMS.db which uses a page size of 4096 bytes. The Page Type bytes are highlighted in light blue and occur at every fifth byte. The byte highlighted in green (0x00) is the first byte of the sequence that is not one of the page type bytes as described above and therefore indicates that there are no more records stored in this pointer map as can be seen within the highlighted darker blue area.

Extrapolating the database size from the Pointer Map page
If you count the Page Type bytes highlighted within Figure 1 in light blue you will find there are 22. This is because we have 22 pages following the Pointer Map page and therefore require 22 records. This allows us to conclude that there are 24 pages in total within this database (page 1, the Pointer Map page and then the 22 pages). By examining the 2 byte integer located at an offset of 16 bytes from the beginning of the database file we have determined that the page size within this database is 4096 bytes. 24 multiplied by 4096 equals 98304. The file size of this particular database is therefore 98,304 bytes which can also be seen within Figure 1.

Carving Considerations
To carve auto vacuum capable databases the following steps would be needed:

• Identify first page of database by detecting the SQLite format 3 header
  
• Establish page size by reading the 2 bytes at offset 16 as a 16 bit integer big endian
• Check the 4 byte 32 bit integer at offset 52 for a non zero value indicating that the database is auto vacuum capable
  
• Go to page 2 of the database at Offset page size
  
• If value is 0x01 or 0x02 or 0x03 or 0x04 or 0x05 set a counter to 1
  
• Move five bytes forward and if value is 0x01 or 0x02 or 0x03 or 0x04 or 0x05 increment the counter by 1
  
• or if the value is not 0x01 or 0x02 or 0x03 or 0x04 or 0x05 begin the calculation of database file size using the formula
  
  
• database size = (counter value + 2) x page size
  

The above discounts the possibility of their being more than one pointer map

page

. Some additional logic would be needed to cater for this eventuality. Pointer map

pages

may contain

page size/5

records. If the counter increments to a point where it equals this value it would be necessary to locate the next pointer map

page

using the formula:

next pointer map page number = (page size/5) + 2 + number of existing pointer map pages.

To calculate the offset to this

page

use the formula:

offset = (page number-1) x page size.

References
http://www.sqlite.org/fileformat.html
http://www.sqlite.org/fileformat2.html
http://www.sqlite.org/src/artifact/cce1c3360c

SQL Server Migration Assistant (SSMA) v5.0 is now available. SSMA simplifies database migration process from Oracle/Sybase/MySQL and Microsoft Access to SQL Server and SQL Azure. SSMA automates all aspects of migration including migration assessment analysis, schema and SQL statement conversion, data migration as well as migration testing to reduce cost and reduce risk of your database migration project.

The new version of SSMA – version - is available with the following key enhancements:

• Support for migration to SQL Server “Denali”
• Multi-thread data migration for improved scale and performance.
• Globalization support for migrating non-English databases to SQL Server.
• Support for installation for SSMA Extension Pack on clustered SQL Server environments.
• Sybase migration enhancement support:
• • Support for migration to SQL Azure
  • Extended data access connectivity to Sybase ASE ADO.NET and ASE ODBC providers
  • Support for proper conversion of case sensitive Sybase database to case sensitive SQL Server
  • Extended support for conversion of Non-ANSI joins for DELETE and UPDATE statements
  • Removed dependency on separate SYSDB database on target SQL Server
• Report use of Oracle’s User Defined Type in the database object and inside PL/SQL.

This has been sitting on my hard drive for a long time. Long story short, the report I posted at Permanent Link to Audit Collection Services Database Partitions Size Report had a couple of bugs:

1. it did not consider the size of the dtString_XXX tables but only the size of dtEvent_XXX tables – this would still give you an idea of the trends, but it could lead to quite different SIZE calculations
2. the query was failing on some instances that have been installed with the wrong (unsupported) Collation settings.

I fixed both bugs, but I don’t have a machine with SQL 2005 and Visual Studio 2005 anymore… so I can’t rebuild my report – but I don’t want to distribute one that only works on SQL 2008 because I know that SQL2005 is still out there. This is partially the reason that held this post back.

Without waiting so much longer, therefore, I decided I’ll just give you the fixed query. Enjoy

 

--Query to get the Partition Table  --for each partition we launch the sp_spaceused stored procedure to determine the size and other info    --partition list  select PartitionId,Status,PartitionStartTime,PartitionCloseTime   into #t1  from dbo.dtPartition with (nolock)  order by PartitionStartTime Desc       --sp_spaceused holder table for dtEvent  create table #t2 (      PartitionId nvarchar(MAX) Collate SQL_Latin1_General_CP1_CI_AS,      rows nvarchar(MAX) Collate SQL_Latin1_General_CP1_CI_AS,      reserved nvarchar(MAX) Collate SQL_Latin1_General_CP1_CI_AS,      data nvarchar(MAX) Collate SQL_Latin1_General_CP1_CI_AS,      index_size nvarchar(MAX) Collate SQL_Latin1_General_CP1_CI_AS,      unused nvarchar(MAX) Collate SQL_Latin1_General_CP1_CI_AS      )    --sp_spaceused holder table for dtString  create table #t3 (      PartitionId nvarchar(MAX) Collate SQL_Latin1_General_CP1_CI_AS,      rows nvarchar(MAX) Collate SQL_Latin1_General_CP1_CI_AS,      reserved nvarchar(MAX) Collate SQL_Latin1_General_CP1_CI_AS,      data nvarchar(MAX) Collate SQL_Latin1_General_CP1_CI_AS,      index_size nvarchar(MAX) Collate SQL_Latin1_General_CP1_CI_AS,      unused nvarchar(MAX) Collate SQL_Latin1_General_CP1_CI_AS      )      set nocount on    --vars used for building Partition GUID and main table name  declare @partGUID nvarchar(MAX)  declare @tblName nvarchar(MAX)  declare @tblNameComplete nvarchar(MAX)  declare @schema nvarchar(MAX)  DECLARE @vQuery NVARCHAR(MAX)    --cursor  declare c cursor for       select PartitionID from #t1  open c  fetch next from c into @partGUID    --start cursor usage  while @@FETCH_STATUS = 0  begin    --tblName - first usage for dtEvent  set @tblName = 'dtEvent_' + @partGUID    --retrieve the schema name  SET @vQuery = 'SELECT @dbschema = TABLE_SCHEMA from INFORMATION_SCHEMA.tables where TABLE_NAME = ''' + @tblName + ''''  EXEC sp_executesql @vQuery,N'@dbschema nvarchar(max) out, @dbtblName nvarchar(max)',@schema out, @tblname    --tblNameComplete  set @tblNameComplete = @schema + '.' + @tblName    INSERT #t2       EXEC sp_spaceused @tblNameComplete                        --tblName - second usage for dtString  set @tblName = 'dtString_' + @partGUID    --retrieve the schema name  SET @vQuery = 'SELECT @dbschema = TABLE_SCHEMA from INFORMATION_SCHEMA.tables where TABLE_NAME = ''' + @tblName + ''''  EXEC sp_executesql @vQuery,N'@dbschema nvarchar(max) out, @dbtblName nvarchar(max)',@schema out, @tblname    --tblNameComplete  set @tblNameComplete = @schema + '.' + @tblName    INSERT #t3       EXEC sp_spaceused @tblNameComplete                      fetch next from c into @partGUID  end  close c  deallocate c      --select * from #t2  --select * from #t3      --results  select #t1.PartitionId,       #t1.Status,       #t1.PartitionStartTime,       #t1.PartitionCloseTime,       #t2.rows,      (CAST(LEFT(#t2.reserved,LEN(#t2.reserved)-3) AS NUMERIC(18,0)) + CAST(LEFT(#t2.reserved,LEN(#t2.reserved)-3) AS NUMERIC(18,0))) as 'reservedKB',       (CAST(LEFT(#t2.data,LEN(#t2.data)-3) AS NUMERIC(18,0)) + CAST(LEFT(#t3.data,LEN(#t3.data)-3) AS NUMERIC(18,0)))as 'dataKB',       (CAST(LEFT(#t2.index_size,LEN(#t2.index_size)-3) AS NUMERIC(18,0)) + CAST(LEFT(#t3.index_size,LEN(#t3.index_size)-3) AS NUMERIC(18,0))) as 'indexKB',       (CAST(LEFT(#t2.unused,LEN(#t2.unused)-3) AS NUMERIC(18,0)) + CAST(LEFT(#t3.unused,LEN(#t3.unused)-3) AS NUMERIC(18,0))) as 'unusedKB'  from #t1  join #t2  on #t2.PartitionId = ('dtEvent_' + #t1.PartitionId)  join #t3  on #t3.PartitionId = ('dtString_' + #t1.PartitionId)  order by PartitionStartTime desc        --cleanup  drop table #t1  drop table #t2  drop table #t3

According to Brief Mobile, Sprint may be revamping its mobile broadband data pricing schema. The report indicates that Sprint mobile broadband devices, including embedded devices like tablets, may get several additional data options starting May 8th. The company will offer a 3GB monthly data bucket for $34.99, a 5GB data bucket for $49.99 and a 10GB data bucket for $79.99. Currently, Sprint offers a 5GB monthly data plan for $59.99 per month — available to both embedded and mobile broadband devices — and a 2GB tablet-only data plan for $29.99 per month; both plans include unlimited 4G connectivity. Sprint declined to comment on the report when contacted by BGR. Read

gPodder is a Podcast receiver/catcher written in Python and pyGTK. It manages podcast feeds for you, and automatically downloads all podcasts from as many feeds as you like. If you are interested in Podcast feeds, simply put the feed URLs into gPodder and it will download all episodes for you automatically. If there is a new episode, it will get it for you. It supports download resume, if the server supports it.

Changes: This is a bugfix release, fixing problems when saving episodes to folders. It also removes GStreamer-based track length detection, as it produced crashes for some users. Description parsing and Windows support have also been improved. For upgrades from older installations, the database schema is repaired to allow multiple GUIDs in different podcasts. Updated translations: French, Galician, Norwegian Bokmål, and Spanish.

Release Tags: Stable, Minor

Tags: Communications, File Sharing, Desktop Environment, GNOME, tools, multimedia, Sound/Audio, Speech, Internet, Web, Dynamic Content

Are you looking to join the mobile revolution? mFoundry is hiring! We take pride in our employees and believe in hiring the very best!

mFoundry is seeking a Data Warehouse and Reporting Engineer to work with our Product Development team. The ideal candidate would be a seasoned data warehouse and reporting engineer. This position is based in Larkspur, CA.

Responsibilities:

• Develop and support large scale Data Warehouse
• Design database schema and data models in close cooperation with the engineering team
• Develop and support Data Warehouse ETL processes
• Develop Analytical Reports against Data Warehouse
• Design and develop analytical applications
• Develop and maintain tools and utilities that assist in testing, deploying and supporting customer implementations
• Design and architect next generation banking application components
• Solve problems and present creative solutions to development team
• Work closely with other developers to design and implement components in timely manner
• Provide accurate work estimates for proposed projects
• Work with QA to ensure high quality of delivered product

Qualifications and Skills:

• Bachelor's degree in Computer Science or related field
• Data Warehouse modeling experience (star schema, snowflake schema, dimensions/facts, etc)
• Extensive experience with SQL queries
• Database experience including MySQL, Postgres, and Oracle
• Database performance optimization
o Experience implementing database partitioning and materialized views a plus.
• Business Intelligence tools experience
o Experience with data visualization tools such as Tableau or QlikView a plus.
• Pentaho Kettle or other ETL tools
• Experience with Predictive Analytics a plus
• Understanding of Object Oriented Design and Design Patterns
• Able to rapidly pick up new concepts; self-motivated and willing to learn new technologies
• Exceptional written and verbal communication skills
• Experience leading remote or offshore resources an asset
• Able to stand up and drive discussions; take ownership and facilitate design meetings, set direction and ensure tasks are complete
• Ability to work in a fast-paced and ever-changing environment
• Must be able to work full time in Larkspur, CA

mFoundry benefit’s include competitive salary and a comprehensive benefits plan (medical, dental, vision, flexible spending account, 401k matching, stock options).

mFoundry is the leading provider of secure mobile financial services to banks, credit unions, merchants and processors. Providing mobile banking technology to Fidelity National Information Systems, First Data Corporation, NCR, the CO-OP Financial Services, and PSCU Financial Services, our award-winning product is used by more than 200 banks and credit unions in the USA.

mFoundry is the only mobile banking provider with a live, nationwide mobile payments solution. As the technical force behind Starbucks Card Mobile, mFoundry leverages its payments knowledge to help financial institutions, payment processors, and merchants create deeper, more valuable customer relationships.

mFoundry is based in the San Francisco Bay Area and has been recognized as one of “10 Emerging Wireless Players to Watch” by IDC as well as the top "Tech Company to Watch" and a three-time top “Innovator” by Bank Technology News. For more information on mFoundry, please visit www.mfoundry.com

For consideration:

• Send resume in MS Word or PDF format to hr@mfoundry.com
• Include cover letter with salary expectations
• The subject line of your email should read: Data Warehouse and Reporting Engineer

mFoundry is an Equal Opportunity Employer. Thank you for your consideration

• Compensation: competitive salary and a comprehensive benefits plan (medical, dental, vision, flexible spending account, 401k matching, stock options)
• Principals only. Recruiters, please don't contact this job poster.
• Please, no phone calls about this job!
• Please do not contact job poster about other services, products or commercial interests.

Der bewährte SSMA (SQL Server Migration Wizard) wurde ein weiteres mal verbessert und erweitert und liegt jetzt in der Version 5 vor. Der SSMA unterstützt und erleichtert erheblich die Datenbankmigration von Oracle, Sybase, MySQL oder Microsoft Access auf eine anständige Datenbank – sprich SQL Server . Der kostenfreie Assistent verfügt dazu über ausführliche Analyse-, Migrations-, und Testfunktionalitäten die sowohl die Datenbank Struktur (Tabellen, Views, Stored Procedures, usw.) sowie die Daten analysieren und migrieren können.

SQL Server Migration Assistant (SSMA) v5.0 is now available. SSMA simplifies database migration process from Oracle/Sybase/MySQL and Microsoft Access to SQL Server and SQL Azure. SSMA automates all aspects of migration including migration assessment analysis, schema and SQL statement conversion, data migration as well as migration testing to reduce cost and reduce risk of your database migration project.

Die Erweiterungen im Detail:

• Support for migration to SQL Server “Denali”
• Multi-thread data migration for improved scale and performance.
• Globalization support for migrating non-English databases to SQL Server.
• Support for installation for SSMA Extension Pack on clustered SQL Server environments.
• Sybase migration enhancement support:
  • Support for migration to SQL Azure
  • Extended data access connectivity to Sybase ASE ADO.NET and ASE ODBC providers
  • Support for proper conversion of case sensitive Sybase database to case sensitive SQL Server
  • Extended support for conversion of Non-ANSI joins for DELETE and UPDATE statements
  • Removed dependency on separate SYSDB database on target SQL Server
• Report use of Oracle’s User Defined Type in the database object and inside PL/SQL.

Weitere Infos und Download des SSMA: http://www.microsoft.com/sqlserver/en/us/product-info/migration.aspx.
Allgemeine Infos zum SSMA (aus einem älteren Blog Artikel): http://blogs.technet.com/b/ptsaustria/archive/2010/01/21/migration-von-access-oracle-mysql-auf-sql-server-azure-migrieren.aspx.

Martin Pöckl
martin.poeckl@microsoft.com

About Us

Join the #1 brand and top Web property focused on answering people’s questions. At Ask.com, we’re building a next-generation Q&A; service – spanning both the Web and mobile devices -- that combines the power of search with insight from real users. Our proprietary algorithms crawl the web for question and answer pairs (400 million and growing), fueling one of the biggest Q&A; databases on the Web. This unique technology, combined with answers from real users in our community, makes Ask.com the most comprehensive Q&A; service around. At Ask, we empower each individual to succeed, communicate their ideas, and take action. When you work here, you know your contributions will have a direct and immediate impact on the company. We are serious about creating and maintaining a company culture that is both innovative and fun, and we’re looking for great people to come join our team.

Summary

Ask.com is looking for experienced MySQL database administrators interested in supporting the next generation answers platform. Be a part of a team taking search to the next level by harnessing the powers of structured data, web extraction technologies and community-driven content. The ideal candidate will have specialized knowledge in database management and tuning for high-performance web applications; expertise in MySQL administration; real-world experience maintaining systems requiring high availability, fault tolerance, and secured data.

Responsibilities

• Setup and maintain production MySQL instances
• Monitor and tune the performance of multiple MySQL servers
• Monitor database events, troubleshoot and resolve problems
• Monitor database backup and validate recovery plans
• Work with development teams for schema design and query tuning
• Design database security roles; enroll users and audit database security
• Follow change management policies to manage change in database environments
• Coordinate with system administration and application development teams for capacity planning and application design
• Cross-train the Site Production team in running and maintaining production databases

Required Experience

• Installing, supporting and maintaining MySQL in a production environment
• Experience with MyISAM, InnoDB and other storage engines
• Basic SQL query and schema optimization
• Hands on experience with MySQL replication, backup and recovery
• MySQL performance tuning
• Identifying problem queries and reporting to the development teams
• Experience with a scripting language such as bash or python
• Deploying schema changes to production

Desirable Experience

• Experience running MySQL on Amazon RDS
• Understanding of Nagios/Cacti/Zenoss monitoring system is a plus
• Understanding of SQL and NoSQL databases

Technical Skills

• 2 years (recently) of production MySQL administration responsibilities
• 2+ years working with the SQL
• 2 years of Linux server experience

Professional Skills

• Self starter with a strong work ethic

Location: Downtown Oakland (Near 12th St. BART Station)

Interested? Go here to apply: http://hire.jobvite.com/j/?cj=o38JVfwz&s;=craigslist

Ask.com is an operating business of IAC (NASDAQ: IACI).


Ask.com does not accept unsolicited resumes from recruitment agencies. Accordingly, agencies are requested not to submit such resumes to our job postings, our employees, or any other part of our company. Recruitment agencies that nonetheless submit unsolicited resumes to Ask.com agree and acknowledge that the company is free to use them at its discretion, including without limitation directly contacting the candidate, and will not under any circumstances be responsible for any recruitment or similar fees related to such unsolicited resumes.

• Compensation: Competitive
• Principals only. Recruiters, please don't contact this job poster.
• Please, no phone calls about this job!
• Please do not contact job poster about other services, products or commercial interests.

Nuestro buen amigo JC Olamendy ha escrito un interesante artículo, aquí los detalle:

Handling disconnected Crud operations using Linq to SQL

Introduction

I’ve been searching in Internet for articles about Linq to SQL and most of the articles are written for connected environments where we have a data context created and the database operations are executed in the data context, but there are common business scenario where we gather data from a context and we move to another context to update the changes to the database such as ASP.NET Web applications and data transfer using Web services as well as desktop applications when we separate the data access code from the business logic code. In this example, I will show how to execute Crud operations using Linq to SQL in a disconnected environment where we use different data contexts to execute different operations. Our application is a classic multi-layer application where we separate the objects in layers according to the functions in the application. In order to simplify the logic, we are going to build a desktop application with only two layers: one for the presentation of the data and the other one for implementing the data access logic.

Getting started with the solution

Let’s open the Visual Studio.NET 2008 IDE and create a Windows Forms Application project. We’re going to create a desktop application which connects to a SQL Server 2005 database system and manages the data on the dbo.Department table on a TestDB database. The schema of the dbo.Department table is show in the Listing 1.

CREATE TABLE [dbo].[Department] ( [DeptID] [int] NOT NULL, [DeptName] [varchar](50) NOT NULL, [Location] [varchar](50) NOT NULL, CONSTRAINT [PK_Department] PRIMARY KEY CLUSTERED ( [DeptID] ASC )WITH (IGNORE_DUP_KEY = OFF) ON [PRIMARY] ) ON [PRIMARY] GO

Listing 1

Then add a new form for the management of the Department entities in the database and add three label controls, three textbox controls, and four button controls in order to get a GUI as shown in the Figure 1.

Figure 1

Next step is to add a new “Linq to SQL Classes” artifact to implement the data access logic through “Linq to SQL” mechanism (see Figure 2).

Figure 2

ver el articulo completo aqui.

Saludos

Français

English

Cette vidéo montre la création d'un serveur SQL Azure, la création d'une base, la possibilité d'utiliser un outil Web (Database Manager) ou un outil à demeure (SQL Server Management Studio) pour y accéder. On y voit également comment charger la base ainsi créée avec le schéma et les données d'une base de données SQL Server disponible à demeure, en utilisation SQL Azure Migration Wizard (http://sqlazuremw.codeplex.com).

This video shows how to create a SQL Azure server, create a SQL Azure database, how to use a web tool (Database Manager) or an on premise tool (SQL Server Management Studio) to access it. It also shows how to upload the so created SQL Azure DB with its schema and its data from an on premise SQL Server DB, by using the SQL Azure Migration Wizard tool (http://sqlazuremw.codeplex.com). NB: the video comments are in French

Benjamin

XMLFox is a graphical XML editing and validation tool for creating valid, well-formed XML documents and/or XSD Schema. It includes an XML View, XML Tree, XML Grid, and XML Script modes for editing, a....

XMLFox Advance is a XML editing and validation tool for creating valid well-formed XML documents and/or XSD Schema. It offers an XML preview as well as XML Tree, XML Grid, and XML Script view of your....

Hi All, I have been using eclipse and MS visual studio .net. I am looking for a way to generate a schema based on the input xml. Visual studio generated as shown below. However, is there a way to mention the below properties through GUI. properties

If you like to use the LINQ to SharePoint provider to perform your SharePoint data queries in server-side code, sooner or later you will encounter a certain tension between the strongly typed world of LINQ queries and the decentralized nature of SharePoint. To enable LINQ querying, source code must include an object-relational mapping that turns lists and content types into classes and turns fields into strongly-typed properties of the content type classes. This code is generated by the SPMetal tool. But website owners can add new lists after your solution has been deployed. The new lists are not accounted for in the object-relational mapping. Similarly, new fields can be added to sites and lists. These are also not accounted for in the object-relational mapping.

Of course, if your LINQ queries and data changes only reference the lists, content types, and fields that were present when the object-relational mapping was generated at design-time, the presence of new entities does not cause a problem. But sometimes a mismatch between the design-time object-relational mapping and the runtime reality of the website content that is being queried can cause problems for your LINQ work. Many of these kinds of problems can be solved by extending the object-relational mapping using the techniques described in Extending the Object-Relational Mapping. In this post, I want to describe a problem, and suggest a solution, that is not mentioned in the latter article.

When SPMetal generates code for a Choice field, it will invent an enum type to represent the possible values of the field. For example, the following code is generated for the standard Priority field on the standard Tasks list:

public enum Priority : int
{
    None = 0,

    Invalid = 1,

    [Microsoft.SharePoint.Linq.ChoiceAttribute(Value = "(1) High")]
    _1High = 2,

    [Microsoft.SharePoint.Linq.ChoiceAttribute(Value = "(2) Normal")]
    _2Normal = 4,

     [Microsoft.SharePoint.Linq.ChoiceAttribute(Value = "(3) Low")]
    _3Low = 8,
}

But what happens if the list owner adds “(4) Trivial” as an additional possible choice to the definition of the Priority column? For task items that have the new value in their Priority field, a LINQ query will return “Invalid” rather than “(4) Trivial” as the value of the field.

The essence of the problem here is that data schemas that can be changed by end users are inherently weakly typed, but the object-relational mapping is imposing a strong-typing straitjacket onto the schema. To solve the problem, we need to loosen that straitjacket; that is, we need to weaken the type of the Choice field from enum to string; but not weaken it too much.

An Easy Solution, But Not Always Applicable

If all you are going to do is query the data, there is a easy solution provided by the configurability of the SPMetal tool. You can change the tool’s default behavior by using a parameters.xml file that is referenced in the command line call to SPMetal. The details of how to do this are in SPMetal and Overriding SPMetal Defaults by Using a Parameters XML File. In this case, you tell SPMetal to treat the Priority field as a string rather than an enum with a parameters.xml file such as the following.

<?xml version="1.0" encoding="utf-8"?>
<Web xmlns="http://schemas.microsoft.com/SharePoint/2009/spmetal">
  <ContentType Name="Task" Class="Task">
    <Column Name="Priority" Member="Priority" Type="String" />
  </ContentType>
  <ExcludeContentType Name="Summary Task"/>
</Web>

Side note: You will notice that I’m also excluding the Summary Task content type from the object-relational mapping, this is to avoid having to do some of the other chores in my solution twice because the Summary Task content type also has the Priority field. 

With this parameters.xml file in use, SPMetal will create the following signature for the Priority property of the Task class:

public String Priority

Instead of the following signature which it would have created by default.

public System.Nullable<Priority> Priority

Side note: The Priority field is also in the standard team website as a site column, so SPMetal will still generate the enum to serve as the type of that site column, but it will not use the enum to be the type of the Priority column in the Task content type.

Now your LINQ queries will return “(4) Trivial” or any other choice that is added to the Priority column definition, rather than “Invalid”.

When You Want to Write to the Choice Field Too

But what if you are going to write data with the LINQ-to-SharePoint provider as well as query it? (You can do that as explained in How to: Write to Content Databases Using LINQ to SharePoint.) Now that you’ve weakened the type of the Priority field, code can write any string to the field, even one that is not included in the column definition’s list of possible choices. You could resolve to always write code that checks the list of possible choices before it writes to a choice field, but you do not want to have to re-write that code every time. One way to make sure your validation executes every time is to put the validating code right in the setter of the property that represents the Choice field.

The first thing you need to do is move the code that declares and implements the property out of the file that SPMetal generates. If you do not do this your customization of the setter in that property would be overwritten whenever SPMetal regenerates the code, and you are almost never in a situation in which you know that you will never have to regenerate the code again.

Fortunately, the classes that SPMetal generates are marked with the partial keyword. That means that you can re-declare the class in another code file and add additional member definitions to the class in the new file.

To continue with the example of the Tasks list and the Priority column,  take these steps:

1. Point SPMetal at site that has a Tasks list and generate your code.

2. Add a new class code file to your VS project. Delete the stub class declaration that is created automatically and delete the namespace declaration, too.

Side note: The code file generated by SPMetal does not declare a namespace, which means the default namespace of the Visual Studio project is assumed. If you have a partial class defined in two code files and one does not put the class inside an explicit namespace declaration, the other code file cannot put the class inside an explicit namespace declaration either, not even an explicit declaration of the default namespace. This seems to be a quirk of the way the compiler deals with partial classes. Hence, you have to delete that explicit namespace declaration that Visual Studio automatically included in the new class code file.

3. Add using statements for Microsoft.SharePoint and Microsoft.SharePoint.Linq.

4. Copy the signature of the Task class declaration from the generated code to your new file and add opening and closing braces . (Do not copy the attributes above the class declaration.)

public partial class Task : Item {

}

5. Copy the backing field for the Priority property in the generated code and paste it into the Task class in your new file. If you have previously applied the Easy Solution above to this project, then the backing field declaration looks like this:

private String  _priority;

Otherwise, it looks like this:

private System.Nullable<Priority> _priority;

6. Move the Priority class declaration from the generated file to the new the same way. In this case you do copy the attribute on the declaration.

[Microsoft.SharePoint.Linq.ColumnAttribute(Name = "Priority", Storage = "_priority", FieldType = "Choice")]
public System.Nullable<Priority> Priority
{
    get {
   
        return this._priority;
    }

    set {

        if ((value != this._priority)) {

            this.OnPropertyChanging("Priority", this._priority);
            this._priority = value;
            this.OnPropertyChanged("Priority");
        }
    }
}

Again, the type of the property will be String if you’ve already applied the Easy Solution to this project.

7. If you have not previously applied the Easy Solution to this project, you now need to change “System.Nullable<Priority>” in both places where it appears in your new file to “String”.

8. You cannot declare the same property and backing field in both files. Unlike the classes, properties cannot be marked “partial”. So, you need to ensure that they are removed from the generated code file and not regenerated on subsequent runs of SPMetal. To accomplish that, create a parameters.xml file with the following content.

<?xml version="1.0" encoding="utf-8"?>
<Web AccessModifier="Internal" xmlns="http://schemas.microsoft.com/SharePoint/2009/spmetal">
  <ContentType Name="Task" Class="Task">
    <ExcludeColumn Name="Priority" />
  </ContentType>
  <ExcludeContentType Name="Summary Task"/>
</Web>

9. Rerun SPMetal with a command line that references your new parameters.xml file. The new generated code file will no longer contain the declarations of the Priority property or its backing field.

Now you need to add your validation logic to the setter of the Choice field class. To continue the example, take these steps.

1. Add the following code to the Priority property setter just above the “if’ statement:

using (SPSite siteCollection = new SPSite(----- ????? -----))
{
    using (SPWeb website = siteCollection.OpenWeb(---- ????? -----))
    {
        SPList taskList = website.GetList(this.Path);        
        SPFieldChoice priorityField = taskList.Fields["Priority"] as SPFieldChoice;

        if (!priorityField.Choices.Contains(value))
        {
            throw new ArgumentOutOfRangeException("value", String.Format("'{0}' is not a possible value for the Choice field 'Priority' in the 'Tasks' list.", value));
        }
    }
}

Working from the bottom up, note the following about this code. If the value that calling code is passing is not in the Choices collection of the SPFieldChoice object, then an exception is thrown. The reference to the field object is obtained through the Fields collection of the SPList object, and a reference to the latter object, in turn, is obtained through the GetList method of the SPWeb object and the Path property of the Task object.

Before any of this can be done, however, the code needs to get references to the SPWeb and SPSite objects. If the code is running where there is an HTTP Context, and thus a non-null SPContext object, your code may be able to get current SPSite and SPWeb objects from the SPContext object. But let’s try to write code that can be used even in a console application or other scenario in which there is no HTTP Context.

2. To get a reference to the SPWeb object, we need a site-collection-relative URL for the website. You can construct this URL by starting with the Path property of the Task object and trimming off from the end of it the website-relative URL of the list. So add the following lines above the “using (SPWeb …” line:

String webRelativeListURL = "/Lists/Tasks";
String siteRelativeWebURL = this.Path.Remove(this.Path.Length - webRelativeListURL.Length);

3. Insert siteRelativeWebURL as the parameter to the OpenWeb method.

You need the absolute URL of the parent site collection to get a reference to the SPSite object. This is harder. The Task class’s Path property does not include the protocol or domain part of the site collection URL. Nor is this information in any member of the class that SPMetal generates. What you need to do is create a member of the Task class that can hold this information and then initialize that member when the Task object is created. To continue the example, take the following steps:

1. Add the following declaration of an internal field to the Task class in your code file (not the generated code file).

internal String parentDataContextWebURL;

You’ll see in a moment why we are giving it that name. For now just note that it is going to hold the URL that is needed by the SPSite constructor.

2. Replace the parameter “----- ????? -----” in the SPSite constructor with the name of your new field parentDataContextWebURL.

The content type class has a constructor that is generated by SPMetal and this is one part of SPMetal behavior that you cannot turn off or change. So you cannot customize this constructor to initialize your new internal field (because  you customization would be overwritten the next time the code is regenerated). Moreover, creating another constructor in your own partial definition of the class would serve no purpose. This is because your calling code does not directly construct the list object and the items within it. Rather these entities are created by the GetList method of the DataContext object. And that method, in turn, calls the generated constructor – the one you cannot modify.

You will notice in the generated code that the content type class constructor does call the partial method OnCreated, so you could give this method an implementation in your own partial class definition. Unfortunately, this method takes no parameters (nor does the constructor that calls it) so there is no way to pass it the data it would need to initialize your new field and, in this scenario, we are deliberately avoiding use of the context.

What you need to do is override the GetList method of the DataContext object so that it initializes the new field on each list object before it returns the list. You can do this because the DataContext-derived class that SPMetal generates is also marked partial. To continue the example, take the following steps:

1. Near the very top of the generated code file is the declaration of the DataContext-derived class. Copy that declaration to the new code file you created earlier and then add the opening and closing braces. The name of the class is determined by what you used as the value of the /code parameter on the command line call of SPMetal. 

public partial class LinqChoiceFieldExperimentsDataContext : Microsoft.SharePoint.Linq.DataContext {

}

2. Add the following override of the DataContext.GetList method to the class.

public override EntityList<T> GetList<T>(string listName)
{ 
    EntityList<T> list = base.GetList<T>(listName);

    if (typeof(T).Name == "Task")
    {          
        foreach (T t in list)
        {
            Task task = t as Task;
            task.parentDataContextWebURL = this.Web;                            
        }                           
        return list;
    }
    return list;
}

Note that the method is passing the DataContext object’s Web property (which holds the object’s absolute URL) to the internal field of each Task object in the list. This field, in turn, is used by your customized setter to create an SPSite object. This URL, by the way, might be the URL of a subsite of the site collection. That does not cause any problems. The SPSite constructor is smart enough to know that what you really want is the parent SPSite of whatever is at the URL in the parameter.

That does it. Calling code like the following will be blocked from writing invalid values to the Choice field.

using (LinqChoiceFieldExperimentsDataContext lCFDC = new LinqChoiceFieldDataContext(http://Contoso/Marketing/))
{
    EntityList<Task> tasks = lCFDC.GetList<Task>("Tasks");
    tasks.First().Priority = "(4) Trivial";
    lCFDC.SubmitChanges();
}

Of course, in its present version, this code only works for the Priority column of the Tasks list.  Consider generalizing it to deal with all Choice fields on all lists. As my college math textbooks used to say, “this is left as an exercise for the reader”.

Integration layer making it possible to load schema definitions from XML

SqlAlchemy schema display script