Author: Kohei Yoshida

LibreOffice contributor, spreadsheet nerd, and machine learning beginner.

This is valid C++ code?

Kohei Yoshida1 Comment

My compiler reported a build error in the following code block today.

long ScDPOutput::GetHeaderDim( const ScAddress& rPos, USHORT& rOrient )
{
    SCCOL nCol = rPos.Col();
    SCROW nRow = rPos.Row();
    SCTAB nTab = rPos.Tab();
    if ( nTab != aStartPos.Tab() )
        return -1;                                      // wrong sheet
 
    //  calculate output positions and sizes
 
    CalcSizes();
 
    //  test for column header
 
    if ( nRow == nTabStartRow && nCol >= nDataStartCol && nCol < nDataStartCol + nColFieldCount )
    {
        rOrient = sheet::DataPilotFieldOrientation_COLUMN;
        long nField = nCol - nDataStartCol;
        return pColFields[nField].nDim;
    }
 
    //  test for row header
 
    if ( nRow+1 == nDataStartRow && nCol >= nTabStartCol == nCol < nTabStartCol + nRowFieldCount )
    {
        rOrient = sheet::DataPilotFieldOrientation_ROW;
        long nField = nCol - nTabStartCol;
        return pRowFields[nField].nDim;
    }
 
    //  test for page field
 
    SCROW nPageStartRow = aStartPos.Row() + ( bDoFilter ? 1 : 0 );
    if ( nCol == aStartPos.Col() && nRow >= nPageStartRow && nRow < nPageStartRow + nPageFieldCount )
    {
        rOrient = sheet::DataPilotFieldOrientation_PAGE;
        long nField = nRow - nPageStartRow;
        return pPageFields[nField].nDim;
    }
 
    //! single data field (?)
 
    rOrient = sheet::DataPilotFieldOrientation_HIDDEN;
    return -1;      // invalid
}

In particular, my compiler didn’t like the == (equality operator) in nTabStartCol == nCol in the 3rd if statement block from the top. Looking at the if statement before and after that, you’ll probably say “yeah, looks like that ‘==’ was supposed to be &&, so what’s the surprise?” Well, the thing is, this piece of code has not changed for at least a few years, which means it was compiling just fine up until today (though it may have caused a bug somewhere…). And even today, it compiled fine before I made a few changes that were not related to this method, and I didn’t modify this method itself at all.

I have to wonder, why this code block compiled fine up till today, and what change of mine triggered the compiler to complain all of a sudden if the method itself is unchanged…. :-/

Trip to Japan

Kohei Yoshida

I took one week off early this month to fly back to Japan for my vacation. This was my first time in 4 years that I visited my family in Japan, and I spent most of my time in my hometown Niigata. It is so weird that, while certain things were just the way I remembered, other things looked so different from what I had in my mind in all these years of being away from the country.

The thing I really enjoyed there was the food! There are many kinds of food I had missed for all these years, but what I missed most was the Mos Burger, a national fast food chain known for its finest quality burgers. While you can find fine quality burgers in other parts of the world (and certainly in the US), Mos Burger has such a distinct and sophisticated taste that in my mind it ranks one of the highest among all the burger stores I’ve tried. So, if you have a chance to travel to Japan, I would certainly recommend you give Mos a try. My personal favorite is the Mos Cheese Burger. Of course, there are other kinds of good foods in Japan such as Ramen noodles and Sushi.

My only regret is that, because I only had one week to spare, I didn’t have time to stop by Tokyo to see folks there. One week was way too short for this. For my next trip back to Japan, I’d like to take at least two weeks off so that I can spend some days in Tokyo.

Anyway, I took a bunch of pictures while I was there, and these are just some of them. I hope you enjoy.


Keisei Rapid Liner from Narita Airport to Tokyo station. While the fastest way to get to Tokyo from Narita is via Narita Express, Keisei Liner is probably a good economical alternative and reasonably convenient. It takes roughly 1.5 hours to get to Tokyo station from the airport without transfer.		 Kendo tournament was taking place in my home town while I was there.
A magnificent view of the sunset at Sea of Japan. The place I stayed at is only 10 minutes walk from the beach. This is one of many infamous vending machines that you see all over Japan. You can think of this as something equivalent of water fountains in the US in terms of its distribution density, though there are certainly more vending machines in Japan per unit area than water fountains in the US.

On the platform for Shinkansen (the Bullet Train) at Niigata station.		 A view from Shinkansen overlooking the outlines of local mountains.

A somewhat busy street in Yokohama. Luckily you don’t need to drive as much in the metropolitan areas thanks to the large train network available throughout the areas.

One thing I’ve noticed is the large number of light-weight vehicles (aka Kei cars) with the engine size less than 660 cc. Since owning a car in Japan is pretty expensive, lots of drivers opt to own a Kei car for its lower cost of ownership.

 A platform at a local station in Yokohama, waiting for my train.

A view of airplane from Terminal 1 at the Narita Airport. I was stranded there for 6 hours since my return flight to Newark was delayed. Continental airline provided a meal coupon to compensate for the delay, though it was a JAL employee who was manning the check-in counter for Continental and gave me the coupon.

Update on scsolver

Kohei Yoshida2 Comments

I’ve made several updates on scsolver that I think is worth mentioning.

First, I’ve finally put an option to solve to a specific value in addition to the minimize and maximize options. But this only works for linear programming models. For non-linear models, if you select to solve to a specific value, you’ll get “Goal not set” error message.

Also, I’ve noticed that the integer constraint option was not working at all; even when this option was selected, the integer constraint was not enforced. This is also fixed. It was just a simple silly mistake I had overlooked which caused this bug.

As far as the non-linear programming backend algorithm goes, I’ve put quite some effort into improving the algorithm a bit. Unfortunately not all of that improvement is available from the UI yet, but the line search algorithm used by the default quasi-Newton algorithm has been switched to a new one that uses quadratic fit line search. This one is a lot more robust than the old algorithm, which was basically a hand-crafted version of golden section search and was quite limiting. The quadratic fit also scales better with search distance. So, hopefully this will improve the overall stability of the algorithm a little. Having said that, writing a robust non-linear algorithm is a huge task, so I have no doubt in my mind that I still have a long way to go.

I will also drop the binary releases of scsolver on Linux, since it’s not easy to build a binary that works universally across different flavors and versions of Linux distros. Also, building from the sources is quite easy on Linux these days. If you have any doubt, see this page for the instructions on how to build it to see if you agree with me on it being “easy”.

Another way to get scsolver, if you are comfortable building the OOo itself from the sources, is to build ooo-build with the --enable-scsolver option. With that, you’ll get the scsolver package already integrated in Calc in place of Calc’s default Solver. So, there is now quite a few ways to get scsolver working on your system. By the way this build option also works on Windows.

Ok, I think that’s all the updates I need to mention. Now, what’s coming next? Well, I have a few things in mind. First, I’d like to re-organize the UI a little, to make it easier and more intuitive to switch between different algorithms. I’m well aware that no single algorithm can solve all models of different shapes and sizes, so I believe it’s important to provide different algorithms for different problem types, especially for non-linear problems. This also includes adding more specialized per-algorithm options to control its run-time behavior.

Another thing I have in mind is some sort of output window to show the running status of the iteration. This is useful for the backend algorithm to give real-time feedback of what’s going on.

I’d also like to do something about the terrible focus problem that the UI is suffering from right now. But quite frankly I don’t have a good idea on how to fix it yet. That may be a limitation of the current UNO AWT API. The worst case, I could reduce the number of child dialogs or avoid having multiple dialogs altogether. Anyway, this is in the back of my mind.

Of course, I will continue working on improving the non-linear algorithms. Currently the non-linear algorithms don’t deal with constraint problems yet, but I’ll get to that eventually.

scsolver code documentation

Kohei Yoshida1 Comment

Inspired by the on-line OOo code documentation that Thorsten put together, I have decided to do the same for my humble scsolver project. I’m pleasantly surprised how good of class diagram that doxygen generates with GraphViz, since the last time I tried doxygen the diagram was very minimalistic only because I didn’t have GraphViz available on my system. Still the digram looks a little too noisy for my taste, but it’s very impressive.

Encrypting an Excel document with password from Calc

Kohei Yoshida1 Comment

The feature of exporting an Excel document encrypted with a password has landed! Now you can save a spreadsheet document as an Excel document with password protection and open it in Excel with that password. The proof is in the following screenshot (the colored emphasis is mine):

Saving a Calc document as Excel with password protection (with encryption)

This feature is now available in the go-oo version of OOo. This also means that, when you edit & save an existing Excel document that has already been encrypted, it will get saved encrypted. Previously when you did this, Calc would save it un-encrypted and you would lose your password protection, which was not good and was a deal breaker for a certain segment of users.

The upstream effort of this feature is underway in the scsheetprotection02 CWS. That CWS also contains another enhancement for sheet and document structure protection (sounds similar but totally unrelated to this document encryption feature), and as soon as I take care of upstreaming that enhancement, I can push the whole CWS for upstream integration. The bad news is that, because that requires a change in the ODF file format & UI change, it will probably take some time before it can be integrated upstream. But I’m making slow but steady progress there, so I’ll keep you guys updated.

Meanwhile, please test the go-oo version for this encryption feature to see if there is any document that Calc fails to encrypt properly (that means Excel can’t open it), and report any bugs to us. I’ve done my own testing, but it’s never as good as many other users testing it. So, thanks!

P.S. Actually this feature has been available for at least two months in the 3.0 branch, but I wanted to see this bug fixed before writing a blog about this so that I could take a pretty screenshot with all texts displayed.

callcatcher

Kohei Yoshida

I just had the chance to play with Caolan‘s cool callcatcher program to search and destroy (or comment out rather) lots of unused methods in Calc’s code. It was surprisingly simple to install and use it. All I had to do after downloading the source package was to

tar xvf callcatcher-1.0.8.tar.gz
cd callcatcher-1.0.8
sudo ./setup.py install

then replace all gcc and g++ calls with callcatcher gcc and callcatcher g++, respectively, to catch all defined functions and their references (or lack thereof). That’s all. Once done, run callanalyse [executable or shared library] to generate the summary output of all unused methods. It was pretty accurate, and as he claims there was no false positives as far as I could tell.

New sheet protection dialog

Kohei Yoshida

I’ve just finished designing a new dialog for Calc’s sheet protection functionality to allow optional sheet protection options. This was actually my first time designing a dialog from scratch instead of modifying an existing one, so I had to dig around and figure out how to add a dialog. It turns out that it is actually very simple once you know what to do. After several hours of creative designing process, I’ve come up with something I can show to people. So here it is:

sheet protection dialog screenshot

One thing to note: obviously this dialog is inspired by the similar functionality offered by Excel, and Excel provides many more options for sheet protection than just the two I’m showing here. The reason I only have two at the moment is because I’ve only implemented support for those two options in Calc core. When we support more options in the core, we can easily add them to the dialog.

This work is on-going in scsheetprotection02 CWS. Aside from the new dialog and sheet protection options, this CWS contains my other work on the binary Excel export encryption as well as sheet and document password interoperability between Excel and Calc. I’m trying to wrap this up, so hopefully I can come up with something that people can try out soon.

OOo class diagram

Kohei Yoshida5 Comments

For those of you who missed my announcement on the go-oo development list, here is the OOo class diagram that I’ve been working on ever since I started hacking the OOo code base who know how many years ago. It’s an Draw document, so you should be able to open this in OOo. I’ve also made a pdf version of it as well.

It’s loosely UML-like, with my own conventions and notes thrown in. I started this to supplement my code-reading process, as drawing a digram helps me to digest the code I’ve just read, and also to refresh my memory on the second visit of the same code. Each page represents an individual module, although I had to break up the sc module into several pages as it grew too large to fit into a single page. Styles are used to classify each drawing objects.

Anyway, I’ve decided to make it publicly available in hopes that someone else will find it useful. Admittedly it’s no way near being complete since I have not visited every corner of OOo’s code base (no one has), but I’m constantly expanding this diagram as I read more and more OOo code.

Want a sneak peak? Here is a screenshot of the sc module class diagram (click on the image to see it at a larger scale):
sc module diagram (thumbnail)