Comments on: DBF import performance http://kohei.us/2009/12/11/dbf-import-performance/ Kohei Yoshida's Webspace Thu, 15 Mar 2012 23:34:39 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Andreas Säger http://kohei.us/2009/12/11/dbf-import-performance/comment-page-1/#comment-11657 Andreas Säger Fri, 07 Jan 2011 20:57:51 +0000 http://kohei.us/?p=589#comment-11657 Hello Kohei, You wrote: > having Base support CSV file as an editable data source will not be trivial, You can use Base and its underlying HSQLDB as csv editor: http://user.services.openoffice.org/en/forum/viewtopic.php?f=83&t=23260 Hello Kohei,
You wrote:
> having Base support CSV file as an editable data source will not be trivial,

You can use Base and its underlying HSQLDB as csv editor: http://user.services.openoffice.org/en/forum/viewtopic.php?f=83&t=23260

]]> By: allbombs http://kohei.us/2009/12/11/dbf-import-performance/comment-page-1/#comment-11009 allbombs Mon, 25 Oct 2010 02:54:59 +0000 http://kohei.us/?p=589#comment-11009 doh..wish the file was public =( doh..wish the file was public

=(

]]> By: Kohei Yoshida http://kohei.us/2009/12/11/dbf-import-performance/comment-page-1/#comment-9895 Kohei Yoshida Wed, 23 Dec 2009 14:19:10 +0000 http://kohei.us/?p=589#comment-9895 Hi Jody, good to hear from you. :-) Unfortunately the file is confidential, but it's just a regular, ordinary-looking dbf file with 24000 rows from column A to CN. About half of the records are string records. Any file that resembles that will do, I think. I just opened up the same file in Gnumeric on my machine, and Gnumeric can open the file in ~11 seconds. So, it does slightly better than Calc even after all my optimizations. Hi Jody, good to hear from you. :-)

Unfortunately the file is confidential, but it’s just a regular, ordinary-looking dbf file with 24000 rows from column A to CN. About half of the records are string records. Any file that resembles that will do, I think.

I just opened up the same file in Gnumeric on my machine, and Gnumeric can open the file in ~11 seconds. So, it does slightly better than Calc even after all my optimizations.

]]> By: Jody Goldberg http://kohei.us/2009/12/11/dbf-import-performance/comment-page-1/#comment-9894 Jody Goldberg Wed, 23 Dec 2009 13:39:22 +0000 http://kohei.us/?p=589#comment-9894 Is the file publicly available? I'd like to see how Gnumeric stacks up. Is the file publicly available?
I’d like to see how Gnumeric stacks up.

]]> By: Kohei Yoshida http://kohei.us/2009/12/11/dbf-import-performance/comment-page-1/#comment-9884 Kohei Yoshida Thu, 17 Dec 2009 03:39:09 +0000 http://kohei.us/?p=589#comment-9884 Hi Jim, As I also said in my reply to your email, having Base support CSV file as an <em>editable</em> data source will not be trivial, to say the least. But I'm not really that much familiar with Base (yet), so you are best advised to ask this on the dev@dba.openoffice.org mailing list. Good luck! Hi Jim,

As I also said in my reply to your email, having Base support CSV file as an editable data source will not be trivial, to say the least. But I’m not really that much familiar with Base (yet), so you are best advised to ask this on the dev@dba.openoffice.org mailing list.

Good luck!

]]> By: Jim Callahan http://kohei.us/2009/12/11/dbf-import-performance/comment-page-1/#comment-9883 Jim Callahan Wed, 16 Dec 2009 14:51:16 +0000 http://kohei.us/?p=589#comment-9883 Terrific! How hard would it be to modify the Calc import module to serve as the Base import module? Some Base tutorials and forums suggest importing CSV files by importing into Calc and copying and pasting to base! Not bad for small tables, but what if more than 64,000 rows? http://sheepdogguides.com/fdb/fdb1imp1.htm http://www.oooforum.org/forum/viewtopic.phtml?t=25434 or raw SQL http://user.services.openoffice.org/en/forum/viewtopic.php?f=83&t=23260 Some likely issues: 1. The modified import module would have to be callable from Base (is there a separate import module in Calc?). 2. The modified import module would have to output to Hyper SQL (HSQLDB) or whatever database Base is using. http://www.hsqldb.org/ a. Database interface http://wiki.services.openoffice.org/wiki/HSQLDB:Tips_and_Tricks http://wiki.services.openoffice.org/wiki/HSQLDB_Integration b. Would have to guess at field types, especially numeric (integer or real?). 3. The modified import module would have to be able to process more than 64,000 rows. I assume this is non-trivial or it would have been done years ago. Thanks, Jim Callahan Orlando, FL Terrific!
How hard would it be to modify the Calc import module to serve as the Base import module?

Some Base tutorials and forums suggest importing CSV files by importing into Calc and copying and pasting to base! Not bad for small tables, but what if more than 64,000 rows?

http://sheepdogguides.com/fdb/fdb1imp1.htm
http://www.oooforum.org/forum/viewtopic.phtml?t=25434

or raw SQL
http://user.services.openoffice.org/en/forum/viewtopic.php?f=83&t=23260

Some likely issues:
1. The modified import module would have to be callable from Base (is there a separate import module in Calc?).
2. The modified import module would have to output to Hyper SQL (HSQLDB) or whatever database Base is using.
http://www.hsqldb.org/
a. Database interface
http://wiki.services.openoffice.org/wiki/HSQLDB:Tips_and_Tricks
http://wiki.services.openoffice.org/wiki/HSQLDB_Integration
b. Would have to guess at field types, especially numeric (integer or real?).
3. The modified import module would have to be able to process more than 64,000 rows.

I assume this is non-trivial or it would have been done years ago.

Thanks,
Jim Callahan
Orlando, FL

]]> By: ervin http://kohei.us/2009/12/11/dbf-import-performance/comment-page-1/#comment-9882 ervin Sat, 12 Dec 2009 08:48:37 +0000 http://kohei.us/?p=589#comment-9882 awesome. awesome.

]]>