Oracle Ireland employee # 74 signing off...

I will shortly be starting my life outside Oracle after some 15 years there. My last day is today.

I've enjoyed it immensely and am proud of our accomplishments. It really doesn't seem like 15 years, and I have been lucky to work on some very exciting projects with some very clever people, many of whom have become friends. I look forward to hearing about all the new releases coming from Database Tools in the future.

Next it is two weeks holidays in France (I hope the weather gets better!) and then the beginning of my next adventure in a new company. More on that later.

I think I'll continue to blog on database tools topics.

Access migration to Application Express without direct SQL Access

I got asked a question recently how to complete an Access migration when you don't have direct SQL access to the Oracle instance where Oracle Application Express is installed (e.g. apex.oracle.com)?

For dealing with the application part, it is not an issue as the Application Migration Workshop feature of APEX (3.0+) allows you to load the results from the Oracle Migration Workbench Exporter for Microsoft Access, so you can capture the meta data for Access Forms and Reports. You can even download a copy of the exporter from the workshop itself.

The challenge is really the schema and data migration part using Oracle SQL Developer (1.2+). By default SQL Developer expects to be able to make a SQL connection to the target Oracle database. However I did think about this use case as we were designing this new Migration Workbench tool. I will describe a solution below.

The only requirement, is that you have SQL access to any Oracle database (9iR2+), because the workbench is driven using an underlying migration repository. You could use the Express Edition of Oracle for this purpose, which is totally free, if you didn't have SQL access to an existing Oracle database.

So let me outline the main steps involved:

1. Start SQL Developer 1.2
2. Make sure you set the following preference: Tools -> Preferences -> Migration -> Generation Options: Least Privilege Schema Migration
   
3. Create a connection to your Access database. Make sure you can browse the tables in the access database and see the data
4. Export the table data to csv format: For each table you want to migrate, use the context menu associated with tables to export as csv format. Make sure you select an encoding that matches your target database. I try to keep everything in UTF-8
5. Create a connection to an Oracle schema.
6. Create a migration repository in this connection. You can do this via the context menu on a connection
7. From your Access connection, context menu, select: Capture Microsoft Access. This will launch the exporter and initiate the capture step of the migration.
8. Take your captured model and now create an Oracle (converted) model by selecting the captured model and via the context menu: Convert to Oracle Model
9. With you converted model, you can now create an object creation script using the context menu: Generate
10. The result of step 9 is presented in a SQL Worksheet, you can edit this to remove objects you are not interested in, then via File -> Save As, save the contents to a SQL file.
11. Login to your APEX Workspace
12. To execute the object creation script you have just created. Goto SQL Workshop -> SQL Scripts -> Upload.
    
13. Once the script is uploaded, View it and select the RUN action. This should create all your schema objects, view the results to make sure all the object were create successfully. You now be able to view these schema objects in the SQL Workshop -> Object Browser.
14. To load our CSV files we will use the Utilities -> Data Load/Unload -> Load, selecting Load Spreadsheet Data. You will do this for each table we want to load data into. Select Load To : Existing Table and Load From: Upload File. You may need to apply appropriate format masks to get the data to load properly.

Notes:

1. You should complete the schema and data migration part of your migration, prior to creating a migration project via the Application Migration Workshop.
2. You may have some post migration cleanup steps, if you had access auto increment columns in your tables, you will need to reset the values of the sequences we have created.
3. Another option to explore depending on your data, would be to export the data from Access tables as SQL INSERT statements, and then it just a simple matter of loading and run that SQL script via apex.

Its summer, time to bring the car out for a good drive

At the weekends, there is nothing better than taking out my Porsche for a long drive drive in the countryside. Much better, than my daily commute! I am a member of the Porsche Club of Ireland, and we had a great drive through the Wicklow mountains recently. The weather wasn't great but the scenery is still dramatic, lots of narrow twisting mountains roads to allow you to eke out the maximum enjoyment. It was fathers day so the kids decided they would both come with me, which was great.

I think we had over 15 members cars out that day. Everybody is very friendly, with a passion for all things Porsche. We started out from the Porsche Center in Dublin and they graciously provided us with coffee before we started and allowed us to gaze over all the new models. Out of my price range of course!

It was a real fun day and I look forward to the next event. We post the pictures from these "drives" on the club web site.

Oracle Database Plugin for the Eclipse Data Tools Platform

As a further commitment to the Eclipse Community, Oracle announces an early adopter release of the Oracle Database Plugin for Eclipse Data Tools Platform. This plugin extends the Eclipse Data Tools Platform to connect to and work with Oracle database objects. The initial plugin supports the ability to connect to Oracle Databases, navigate through all database objects, execute stored procedures and functions, and view textual and graphical execution plans.



This release has been tested against DTP 1.0. We will revise the plugin as required once DTP 1.5 is released as part of the imminent Eclipse Euorpa release.

Oracle has published a Statement of Direction which outlines our future plans.

Feedback from the community is important to us so Oracle has created a forum on OTN to provide a means for this feedback to be collected and allow Oracle to respond. Assuming a positive level of interest from the community, we would seek to formally join the DTP project for the purpose of contributing to the Connectivity, Model Base, SQL Development Tools and particularly the Enablement subproject to provide specialized support for the Oracle Database.

Oracle Database Migration has reached the next level!

SQL Developer 1.2 is now production and with it our new integrated migration workbench. Find out more information on OTN here. Over the last 6 months or so as I have talked about this new product, I used the tag line "Taking database migration to the next level", well... I think we are there now. We are the first database vendor to provide an integrated migration tool into a developers IDE with all of the resulting productivity benefits this brings.

The SQL Developer Migration Workbench is the tool to aid in the migration of third party databases onto the Oracle platform. This tool allows you to migrate your existing Microsoft Access, Microsoft SQL Server and MySQL databases (including schema objects, data and stored procedures/functions) to Oracle.

Features of this production release include:

• Enhanced user interface - This release harnesses the enhanced GUI environment of SQL Developer and works seamlessly with other SQL Developer components. This means that users have one tool to browse third-party databases, migrate selected objects to Oracle and then work with them.
• Existing SQL Developer users should find the Migration Workbench familiar and easy to use.
• Quick Migration Wizard provides the easiest and quickest means of doing a database migration.
  
• Step driven migration offers control at each stage of the migration process.
• Fine grain migration support provides users with the ability to select specific objects for migration.
• Least privilege migration - The ability to migrate objects from source to target without the need for dba rights. The workbench will migrate objects it has rights to view so does not require any special privileges to run.
• Platform Supported:
  
• Supports Microsoft Access versions 97, 2000, 2002/XP, 2003
  
• Microsoft SQL Server 7, 2000, 2005
  
• MySQL versions 3, 4, 5.
• Parallel Online data move - The ability to move the data using a number of parallel connections for increased throughput.
• Offline data move script generation - Generates scripts to allow for export of source data and import to target database of offline data move. We also support offline capture as well.
  
• Language Translation Features - Supports translation of stored programs, procedures, functions, triggers, constraints and views defined in Transact SQL or Microsoft Access SQL.
• Translation scratch editor - An interactive editing facility for Transact SQL and Microsoft Access SQL allowing for instant translation to PL/SQL or SQL. This editor supports both single-statement translation and translation of entire SQL scripts.
• Translation difference viewer - Inline difference viewer for examining translated SQL. This viewer provides color coded side-by-side comparison of translated SQL to display semantic similarities between the source and translated code.
• Any many more interesting features for you to find out about....
  

We have also prepared a number of online demos (viewlets) so you can see this new tool in action. Check them out on OTN here.

I am very proud of what my team has achieved with this release. A lot of work over the past year has gone into it. This is only the beginning, expect further innovative releases from the database tools team in the future.

I would like to thank our early adopters who provided some very constructive feedback. Hopefully you see the results of your feedback in this production release.

Access Migration Tutorial

Are you considering migrating that Microsoft Access application to Oracle Application Express? Well read on...

Hopefully you are aware of Oracle SQL Developer and that we have redeveloped the Migration Workbench and integrated it tightly with Oracle SQL Developer. An early adopter version of this is available now and will be production very soon. More of that in a subsequent post.

With Oracle Application Express 3.0 we introduced the Application Migration Workshop to assist with migrating your Access Forms & Reports. When I talk about this solution, I get asked do we have a step by step guide or methodology for such migrations. So, we have produced a migration tutorial to address this and have published it on OTN.

We have taken the Microsoft Access sample application, Northwind Traders and migrated it to Oracle Application Express. The tutorial covers this in step by step detail. Following this tutorial would be a useful exercise for any user that wishes to undertake migrating their applications from Microsoft Access to Oracle Application Express. We have called the converted application Southwind Wholesalers. :-)

You can see it running on apex.oracle.com and we have also provided it as a packaged application so you can examine it in detail.

Updated Oracle SQL Developer Migration Workbench Early Adopter Release

We have updated the early adopter release of Oracle SQL Developer Migration Workbench today on OTN. You can get it from here. This is our final preview release, before we go production. We are now functionally complete for this initial production release and are now focused on fixing our final "show stopper" bugs. We have had good feedback from our user community via our feedback application and also via the Workbench forum. They have uncovered a number of bugs, most of which we have now addressed (Oracle9i as a repository and Access data migration issues for example) and the remaining ones we will resolve prior to production. I encourage everybody to update to this latest release and continue to provide us with feedback.

I have been working extensively with different builds of the Migration Workbench these past couple of weeks as we closed in on our goal to refresh the early adopter version. In my "biased" opinion is it looking much stronger and I would like to outline some of the new features in this updated early adopter release.

Quick Migrate
In the orginal Migration Workbench we had a wizard driven approach to simplify migrations and I felt it was important to bring this functionality back. With our Quick Migrate wizard, I believe we have improved from the original wizard, since we will leverage our least priviliege migration capabilities, assume sensible defaults and create/remove our migration repository.



So if you have a schema on SQL Server or a single Access mdb file to migrate to an existing Oracle schema, this should be the easiest and quickest migration option for you. Another nice feature, if you are doing an access migration, is that we have added command line support to our exporter so, we will automatically launch the correct Access exporter for the Access connection that you specify.

Offline Capture
This was a popular feature with our consultants and partner technical services folks, with the original Workbench, as it allowed them to work remote from the customer/partner. We have now added back in that feature.

Migration Reports
We have added in some initial migration reports available under Reports->Shared Reports. This will be an area we will add to into the future, as we can mine our rich metadata repository to provide you with useful information. If you have suggestions for additional reports let me know. I will also publish more details about our repository, so you can develop your own migration reports as well. Maybe we should have a competition for the best contributed report? I think we have a couple of 1GB USB keys left over from our Database Developer Day in Dublin I could use as prizes.

Translation Scratch Editor
We have reworked this feature extensively. I originally wanted to add a feature that would enable you to validate our translated SQL. As we worked through different iterations of how best to implement this feature, we came up with the idea about leveraging our existing Worksheet capabilities, which I think is very cool and I am very pleased with how this turned out.


We have also done a lot of work to improve incremental capture and improve our filtering capabilities from our early adopter release. We have integrated our MySQL parser from the original Migration Workbench and will extend its capabilities in subsequent releases to be as functional as our new TSQL parser and also support SQL statement level translation. (workaround for now, within the scratch editor, is just wrap the SQL statement in a procedure). We have also implemented the ability to update your Access mdb file, to create link tables to point to your newly migrated schema. This was also a feature of the original workbench. We hope to add some additional usability tweaks to create an ODBC OSN on the fly and provide a select list of known Oracle DSN. Hopefully that will make it in before production as well.

We have made fixes to ensure correct generation order for pl/sql procedures to resolve dependencies, so more pl/sql procedures should compile correctly first time. We made improvement to handle inline DDL statements correctly. Temporary tables, normal tables and other DDL are lifted out of the body of the procedure/function and are created separately.

Now for the final bug fix push by the development teams in Dublin and Bangalore. Our QA team, have been doing a good job verifying our fixes and closing off our bugs. The teams have been working hard on this for many months now and I believe we are in touching distance of reaching our goal. It will be very exciting for me personally to see this second generation migration tool reach production. We'll all need some time off when this is done to recharge...

Oracle Application Express 3.0 a credible alternative to Microsoft Access?

I hope most people know by now that Oracle Application Express (APEX) 3.0 has gone production, is available for download from OTN and our hosted instance, apex.oracle.com had also been upgraded to 3.0. There has been some positive reviews in the press recently, here are links to a number of them:

Oracle Updates Application Express Tool
Oracle updates free Web development tool
Oracle has big ambitions for Application Express
Oracle improves free tool for building Web Applications
Oracle Application Express 3.0 Touts Access Migration


Most customers I talk with have problems with Access and are looking at alternatives, which I guess is the reason I am talking with them in the first place. However, Microsoft Access is a popular desktop database and was updated recently as part of the Office 2007. That said, it is desktop centric and more suited (optimized?) for productivity applications used by individuals or small groups.

On our OTN page about the Application Migration Workshop we have published a high level feature comparison of Access and Oracle APEX.

Is Oracle Application Express 3.0 a credible alternative to Microsoft Access?

What do you think?

Oracle SQL Developer Migration Workbench Early Adopter Release

Last week we released on OTN the early adopter release of the Oracle SQL Developer Migration Workbench. You can find more details about it here. It was a very important release for us, and marks the start of a new generation of migration tools.

It is nearly 10 years ago when the original Oracle Migration Workbench was released, we supported migrating SQL Server 6.5 to Oracle8 then. At that time, I believe we were the first to introduce a GUI tool. Previously we had provided a series of migration scripts (shell based + SQL) and a stored procedure converter utility. We went on to add support for Access, Sybase, Informix, DB2 utilizing the same user interface by leveraging our plugin architecture. Over the years we have seen our database competitors and others release similar migration tools for their databases.

With this release, I believe we have made the same dramatic shift again that we did back in 1998. By integrating our migration tool as a extension of SQL developer (our very popular tool for database developers) we have provided our users with a modern intuitive UI tightly integrated into an IDE, that should make users even more productive as they carry out database migrations. I don't believe any of our competitors have delivered such tight integration.

This initial release supports Microsoft SQL Server, Access and MySQL. We are introducing support for migrating Microsoft SQL Server 2005 with this release. These third party databases represents the most popular downloads for our existing Oracle Migration Workbench. We will add further platforms in the future. We have also architected this solution, to make it even easier to extend and leverage the rich core migration functionality that we have developed. We hope that others will also extend this tool going forward adding support for additional databases.

The focus now, is on completing some features which missed the cut for the early adopter release, (more on that in a later post) , getting feedback from our user community and fixing as many reported bugs to ensure the highest possible quality release, when we go production, as SQL Developer 1.2. I encourage you to try it out and provide us with feedback. We have setup a comment application which you can provide us with feedback. You can access it here.

Some of my favorite features of this new release includes:

• Least privilege migration - You no longer need dba privileges
• Online Data Move - We have enhanced the online data move and provide parallel data move and the degree of parallelism is configurable
• New T/SQL parser - We have completing rewritten our T/SQL parser. If I'm honest, it was long over due, but this new parser, provides us with the right foundation for a much greater level of automation in converting complex objects (stored procedures, views, triggers)
• Translation Scratch Editor - allows for the instant translation of Transact SQL or Microsoft Access SQL to PL/SQL or SQL.
• Translation Difference Viewer - a color-coded side-by-side viewer to display semantic similarities between the source and translated code.

Looking forward to reviewing the feedback from our user community, getting those missing features completed and getting this new tool to production status as part of SQL Developer 1.2

Consolidation is the key driver for Migration from Microsoft Access to Oracle Application Express

I have just been reviewing the survey results and feedback received from early adopters of the Application Migration Workshop a new feature of Oracle Application Express to assist with migration of Microsoft Access Forms and Reports to Oracle Application Express.

Looking at the data, consolidation is the key reason. In fact, it was twice as popular as the next given reasons of company direction and performance.

So what does consolidation really mean in this context. It is 1 to 1, i.e. one Access database to a unique schema in Oracle or is it a many to 1, removing duplication and increasing the shared access to a "single source of truth". I think it is a mixture of both.

In talking with customers over the last year or so, some customers have told me that they have thousands of access applications and refer to it as their "access problem". Clearly it would be a stretch to think that they have thousands of unique applications, so there must be a lot of duplication and redundant access databases.

So how do you define an appropriate access migration strategy. Well I believe firstly, you need more information about your access databases. This is really an assessment phase. In doing an assessment of a large number of access databases, I would split it into two parts:

1. Technical Assessment
2. Business Assessment

Technical Assessment

In the technical assessment phase, you are trying to build up some key data about these access databases. I refer to it as a technical assessment, as hopefully you could have a utility to collect this information. Data to collect would include:

• Name, Owner, Size
• Date created, Date last modified
• Version
• # of Tables, Forms, Reports, Queries, Pages, Modules,
• Links
• links to other mdb files
• linked data e.g. link tables via ODBC to Oracle, SQL Server, ...
  
• Location (server)
  
• ...

Business Assessment (Business Level Discovery)

This requires human involvement, and is critical for a subsequent analysis phase. Data points here would include:

• Business Owner (by department, group)
• Description (purpose)
• Application Type (data entry, reporting, ...)
  
• Priority (Business Critical to Redundant)
• Action (migrate, ignore, duplicate, remove, ...)
• # of users
  

I can see this being the basis for a nice Oracle Application Express Application, that would support the data entry of business level data, and the generation of useful reports and charts. This would then facilitate the next level of analysis and allow you to ask some interesting questions:

• What is my spread of versions of Microsoft Access?
• How many copies of project.mdb or tasklist.mdb do I have? :-)
• What's my largest / average access database size?
• Which access databases are most heavily used?
  
• ...

Now I think that would be a very useful data collection utility and Oracle APEX application.