How to (pretend to) write an export filter

It turns out that pretending to write an export filter, at least adding a new entry to the Export dialog, is quite easy. In fact, you don’t even have to write a single line of code. Here is what to do.

Suppose you do your own build, and you have installed the OO.o that you have built. Now, go back to your build tree, and change directory into the following location

filter/source/config/fragments

and add the following two new files relative to this location:

./filters/calc_Kohei_SDF_Filter.xcu
./filters/calc_Kohei_SDF_ui.xcu

You can name your files anyway you want, of course. ;-) Anyway, put the following XML fragments into these files:

<!-- calc_Kohei_SDF_Filter.xcu -->
<node oor:name="calc_Kohei_SDF_Filter" oor:op="replace">
  <prop oor:name="Flags"><value>EXPORT ALIEN 3RDPARTYFILTER</value></prop>
  <prop oor:name="UIComponent"/>
  <prop oor:name="FilterService"><value>com.sun.star.comp.packages.KoheiSuperDuperFileExporter</value></prop>
  <prop oor:name="UserData"/>
  <prop oor:name="FileFormatVersion"/>
  <prop oor:name="Type"><value>Kohei_SDF</value></prop>
  <prop oor:name="TemplateName"/>
  <prop oor:name="DocumentService"><value>com.sun.star.sheet.SpreadsheetDocument</value></prop>
</node>
 
<!-- calc_Kohei_SDF_Filter_ui.xcu -->
<node oor:name="calc_Kohei_SDF_Filter">
  <prop oor:name="UIName"><value xml:lang="x-default">Kohei Super Duper File Format</value>
    <value xml:lang="en-US">Kohei Super Duper File Format</value>
    <value xml:lang="de">Kohei Super Duper File Format</value>
  </prop>
</node>

Likewise, create another file:

./types/Kohei_SDF.xcu

with the following content

<!-- Kohei_SDF.xcu -->
<node oor:name="Kohei_SDF" oor:op="replace" >
  <prop oor:name="DetectService"/>
  <prop oor:name="URLPattern"/>
  <prop oor:name="Extensions"><value>koheisdf</value></prop>
  <prop oor:name="MediaType"/>
  <prop oor:name="Preferred"><value>false</value></prop>
  <prop oor:name="PreferredFilter"><value>calc_Kohei_SDF_Filter</value></prop>
  <prop oor:name="UIName"><value xml:lang="x-default">Kohei Super Duper File Format</value></prop>
  <prop oor:name="ClipboardFormat"><value>doctype:Workbook</value></prop>
</node>

Once these new files are in place, add these files to fcfg_calc.mk so that the build process can find them. To add, open fcfg_calc.mk and add Kohei_SDF to the end of T4_CALC, calc_Kohei_SDF_Filter to F4_CALC, and calc_Kohei_SDF_Filter_ui to F4_UI_CALC. Save the file and rebuild the module. This should rebuild the following configuration files (build done on Linux):

./unxlngi6.pro/misc/filters/modulepacks/fcfg_calc_types.xcu
./unxlngi6.pro/misc/filters/modulepacks/fcfg_calc_filters.xcu
./unxlngi6.pro/bin/fcfg_langpack_en-US.zip

One note: the language pack zip package should contain the file named Filter.xcu with the new UI string you just put in. If you don’t see that, remove the whole unxlngi6.pro directory and build the module again.

Now it’s time to update your installation. You need to update the following files:

<install_dir>/share/registry/modules/org/openoffice/TypeDetection/Filter/fcfg_calc_filters.xcu
<install_dir>/share/registry/modules/org/openoffice/TypeDetection/Types/fcfg_calc_types.xcu

with the new ones you just rebuilt. Next, unpack the langpack zip file and extract Filter.xcu. Place this file in

<install_dir>/share/registry/res/en-US/org/openoffice/TypeDetection/Filter.xcu

to replace the old one.

Ok so far? There is one more thing you need to do to complete the process. Since these configuration files are cached, in order for the updated configuration files to take effect, the cached data must be removed. The cached data is in the user configuration directory, so you need to locate and delete the following directory:

rm -rf <user_config_dir>/user/registry/cache

That’s it! Now, fire up Calc and launch the Export dialog. You see the new file format entry you just put in. :-)

Export dialog with new export filter entry

Just try not to export your file using this new filter for real, because that will utterly fail. ;-)

Trip to Prague

I just came back from a week-long trip to the City of Prague – the capital and largest city of the Chech Republic – to participate in Novell developer’s team summit. Just saying that I had such a good time is an understatement. It was a blast! The weird thing is that this was the first time I ever met with anybody who is involved in the OO.o project face-to-face, and instead of text-only communication that we normally conduct, talking with actual voice and seeing their physical face brings such a warm, pleasant feeling to the conversation.

Aside from the meetings we had in the office, we spent the evenings and the Saturday exploring the city. The most memorable moment of course is the “blackout” incident on Thursday.

Here is the story. We went by train to a small town outside of Prague on Thursday evening. The plan was to find a restaurant in that town to sit down, relax, eat and chat (the usual stuff). We got off the train, and headed for the first restaurant closest from the train station, but unfortunately it was closed. But hey, accidents happen all the time, so we immediately regrouped and headed for the second restaurant in town, thinking that the odds of two restaurants being closed were very low.

And guess what, the second restaurant was also closed! Jan, our trusted local guide, sensed that something was wrong, so he found someone local and asked him what was going on. We then found out that there was a power outage in that town, and as a result of that all their local restaurants were forced to close for the evening.

At that point, our only choices were either to wait one hour for the next train, or walk 6 kilometers to the next town and hope that the power outage didn’t reach there. We chose the latter.

Long story short, we ended up walking that long 6-kilometer trail through the woods to the next town, to finally find a restaurant! In retrospect, though, it was probably the best team-building exercise anyone could have come up with (plus a good exercise physically). :-) But I’d rather not go through that again. ;-)

This trip was actually my first visit to Europe, and I’m sure I’ll be back again. It’s sad that we don’t know when we will meet each other again the next time, but hopefully not too distant future.

Last day at SlickEdit

Today was officially my last day at SlickEdit. I was pretty busy all day trying to transfer knowledge to my coworkers so that they can pick up where I left off. I hope I did all I could to minimize disruption as a result of my departure. They were kind enough to do a farewell lunch for me, and even ice cream celebration in the evening! Thanks guys. I’ve really been my pleasure working with you all. I will certainly miss it.

So, starting tomorrow, I will be with Novell, working full-time on OpenOffice.org. I can’t thank Novell enough for giving me this opportunity, and I will do my best to help bring this wonderfull office suite to the next level! We have a lot of work to do, so let’s get started. :-)

Quote of the Day

“Changes in a system can be made in two primary ways. I like to call them Edit and Pray and Cover and Modify. Unfortunately, Edit and Pray is pretty much the industry standard.” — from Chapter 2, Working Effectively with Legacy Code, Michael Feathers

Sorry, I had at really laugh at this one. :-D So true…

Sudoku Solver

I don’t know about the rest of the world, but the part of the world I live in, Sudoku was spreading like a wildfire. It was almost to the point that you couldn’t be in a place where you didn’t see anyone playing Sudoku in one form or another. This trend seems to have died down a bit, but Sudoku still is quite popular puzzle and a good way to spend (waste?) some time exercising your brain cells.

I, on the other hand, have never played Sudoku. But that doesn’t mean that I can’t write a solver for it. It’s written in Python, and is pretty simple and straightforward. It uses a recursive backtracking to find a solution, so there is no fancy algorithm there. Still, it manages to solve most of the Sudoku puzzles I’ve encountered within a second. For some harder ones it spends about a few seconds, but it’s still within what I consider a reasonable time frame.

How to use it is pretty simple. When you download and unpack the file, you should see a file named sdksolve in the base directory. It takes a text file containing a Sudoku grid as an input file, and spits out the solution to standard output. There are some example input files under the example directory, so you can use those to see how it works. Here is an example run:

./sdksolve sample/web1.txt
Sudoku Solver by Kohei Yoshida
--------------------------------------------------------------------
Input Sudoku Grid
--------------------------------------------------------------------
5 * * 3 * * 1 * *
7 4 * 2 * * * 3 6
* * * * 1 * * 2 4
* * * 7 2 * 8 6 *
4 * 2 * * * 7 * 5
* 7 9 * 8 1 * * *
2 1 * * 7 * * * *
3 6 * * * 2 * 5 8
* * 4 * * 6 * * 1
--------------------------------------------------------------------
5 2 6 3 4 9 1 8 7
7 4 1 2 5 8 9 3 6
8 9 3 6 1 7 5 2 4
1 3 5 7 2 4 8 6 9
4 8 2 9 6 3 7 1 5
6 7 9 5 8 1 3 4 2
2 1 8 4 7 5 6 9 3
3 6 7 1 9 2 4 5 8
9 5 4 8 3 6 2 7 1
valid sudoku grid!
number of iterations: 116

So, here goes yet-another-Sudoku-solver out in the wild. Have fun! :-)

Constraint parsing bug

One avid user of my Calc Solver from Belgium found a rather serious, and a little embarrassing bug and brought it to my attention (thank you, Ludovic!). It was the way I convert a constraint formula from the spreadsheet format to the internal model, which utterly fails to do it correctly when the left-hand-side cell contains a constant value. I need to fix this sooner rather than later since I believe this is rather a severe bug that should not be left in the code knowingly (now that I know it…).

So, I’ll spend my next available time trying to fix it, and put it into ooo-build first. Then bring the fix to scsolver02 cws afterwards. I’ll have to hold off the upstream integration work for now.

scsolver02 CWS

Ah, my first CWS. :-) It’s intended for upstreaming the solver code that I’ve been working on for years to the main codebase.

There are still many tasks to be completed however, such as writing a spec file and working out build integration details. Now, that‘s taking more time than actually writing code (exaggerated of course, but it feels that way). The main issue at the moment is how to conditionally show the solver menu item, since it makes no sense to show this menu when the solver is not enabled at build time.
I’m also having a weird problem with the spec file template. Reading through the Basic code indicates that I’m supposed to have additional file called autotext placed in order to design UI? Two hard-coded URLs are found in the code, but neither URLs seem to work. Hmm…

This much has been done so far:

  • created two new modules lpsolve and scsolver and populated them.
  • made a new configure option –enable-scsolver, which is disabled by default.
  • ran build tests three times to make sure it builds, with or without –enable-scsolver.

Upstreaming scsolver

I’m in the process of upstreaming my solver (a.k.a scsolver – its cvs module name) to the OpenOffice.org repository at the moment. So, with everything else put on hold, all my effort is shifting toward making that happen.

Typically, any new code to the OO.o is upstreamed via a process known as the child work space (CWS), which is basically a front-end to the CVS repository (soon to be SVN’ed, I suppose? ;-) ) designed to ease the management of source control from different parties, including external contributors such as myself.

But CWS can be a bit intimidating and confusing to the un-initiated. For instance, it took me a great deal of trouble to figure out how to add new top-level modules. All sorts of emails went back and forth, and in the end, thanks especially to Petr, I was able to at least add new stub modules directories in cvs HEAD, and filed an issue to have the right person (Jens?) add cvs alias to those modules so that they can be checked out for further work.

Also, because my code has dependency on lp_solve, which itself is an external project, the right steps needed to be taken as outlined in the external project page to make sure that we are complying with the license of that particular software. Not sure how long this process will take, but at least I did my part (or did I?).

Anyway, things are moving along. :-) Michael was kind enough to create a new cws for me, so hopefully once those modules get aliased for checkout I’ll be able to start working on putting my code in that cws.

Oh yeah, also not to forget that patch for lpsolve that maho wants to commit. ;-)

Hacking Calc – The First Step

I’m happy to announce that the article I submitted to the OpenOffice.org Developer Article Contest has been selected as June’s winner! You can read my article here. Though this article attempts to cover the hard topic of hacking the core part of OO.o’s code base, I tried to keep it somewhat of an easy read.

I hope this article will help lower the initial barrier to entry, because hacking OO.o, though it admittedly comes with a steep learning curve, is quite fun once you set your foot in the door. So, even if you are a little intimidated by the sheer size of the code base, come and try with your hand a little. You might actually like it. :-)

Interested in writing your own article for the contest? Cool. Go read the guideline and submit your own!

Solver new snapshot available

I’ve uploaded a new snapshot of Solver up on the usual place. The difference from the last snapshot is that the Solver now uses lp_solve as the backend LP optimizer. lp_solve is quite a robust optimizer currently developed and maintained by Kjell Eikland and Peter Notebaert. It has an excellent track record with users across many industries, so if you have a model that my Solver failed to solve previously, try out this snapshot to see if it will do a better job for you.

Now that the UNO package version of my Solver is finally out after a long 4-month hiatus , my next immediate task is to work on integrating this lp_solve-based Solver into ooo-build. For this, I will take a different integration strategy.

To recap, the strategy I took to implement the lp_solve-based Solver (this snapshot) was to add a UNO layer to lp_solve to turn it into a UNO library, and add a UNO hook to the main Solver component to glue the two separate components. This strategy was necessary despite its over-complexity and obvious overhead because, currently, it is not possible to include a shared library into a UNO package unless it provides a UNO service, or the package registration will simply fail. This is because OO.o expects every library included inside a package to export a certain set of callback functions for component registration, or it will balk at it.

The integration into the build, however, will be much simpler because there is no need for component registration. So I can simply have the Solver core library dynamically link to it and avoid the overhead of UNO. The downside of this is that I have to write additional code for it ;-), but I expect it to take only a minimal amount of effort.