26

(2 replies, posted in General)

Dave,

At first, if the SqliteLibrary property is not set, the SqlitePassDatabase Component searchs the default path (in SqlitePassConst.pas) :

{ Default sqlite library file }
{$IFDEF MSWINDOWS}
  DefaultSQLiteLibrary = 'sqlite3.dll';
  SqlitePassPathSeparator = '\';
{$ELSE}
  DefaultSQLiteLibrary = 'libsqlite3.so';
  DefaultSQLitePath = '/usr/local/lib/';
  SqlitePassPathSeparator = '/';
{$ENDIF}

I'm going to work on this in order to provide multiple default search paths. As I'm not a Linux expert, could you tell me the default paths that need to be searched (by priority order...). Thanks

Luc

27

(10 replies, posted in Bug Tracking)

Hi Dave,

Do you have the same problem on Windows ?

If not, first, at designtime, try to connect your DB, then Right Click on SqlitePassDatabase Component. Select "Show Log Error Dialog" in the menu. Move backward (lef arrow) and check the msg list.

You should find a message describing the problem. I guess one sqlite library function is missing. I bet on :

sqlite3_libversion_number, sqlite3_sourceid or sqlite3_compileoption_used ...

Some Linux distribution doesn't have the latest SQLlite library and doesn't always export the above function(s).

If my guess is correct, you have two solutions :

1 - Comment this function in source code in SqlitePassEngine.inc lines 362 +

@SqliteDbv3_SqliteLibVersionNumber := LoadFunction('sqlite3_libversion_number');
@SqliteDbv3_SqliteLibSourceId    := LoadFunction('sqlite3_sourceid');
@SqliteDbv3_compileoption_used   := LoadFunction('sqlite3_compileoption_used');

then rebuild the package

2 - Or install an updated sqlite library that exports the required functions.


Let me know about this quick fix.
Regards

Luc

28

(1 replies, posted in General)

Hi Curt,

- 1 -
Additional translations rules should be set at design time, using the  TSqlitePassDatabase.DataTypeOptions.TranslationRules editor, or the TSqlitePassDatabase.DataTypeOptions.CustomFieldDefs editor if you want to have control on a specific table field.

Another way is to get the OnDataTypeConversion event fired. If you want to, you have to set the detectionMode to dmCustom  AND the TSqlitePassDatabase.DataTypeOptions.LoadOptions to [loTranslationRules,loCustomFieldDefs] and exclude [loDefaultProperties, loCustomProperties] because they might override the dmCustom detectionMode.

Your code is correct to add translations rules at runtime, but it won't work with the current components implementation. I have to add some easy property to execute the datatype mapping on demand, at runtime : next version... smile


- Note -

There is a typo error in TDataTypeConversion prototype (SqlitePassDbo.pas) (FiedDef instead of FieldDef)

TDataTypeConversion = procedure (Database: TSqlitePassDatabase; TableDef: TSqlitePassTableDef;
                                 FieDef -> FieldDef: TSqlitePassTableFieldDef) of object;



- 2 -
The DataTypeOptions.DefaultFieldType is only used when no datafieldtype match was found for a field during the datatype conversion process.
If you want to convert all your field to  ftString, use dmForceStr.

--------

If you don't understand my poor explanation, let me know. I will try to clarify my mind and provide a better answer !

Regards.

29

(6 replies, posted in Miscalleneous posts)

Hi Parcel,

Thanks for these FIX.

I include some of them in the 0.55 release.

Could you tell me where {$DEFINE DELPHI2009PLUS} is located (I didn't mention this define in sqlitepassdbo.inc...).

Regards
Luc

30

(5 replies, posted in General)

Hi,

Thanks for trying the components.

I'm afraid your query is too complex to be correctly handled with the 0.51 package. Give a try with sqlCur.ReadOnly := True without to much hope...

The 0.55 package should solve this issue and will be available for download in a couple of days. So I would like to test your db and querie or a sample before publishing the next release. Could you send a sample by mail ?

Your code is correct. Here are some suggestions

   // creating and connecting to the db
    sqlDB:=TSqlitePassDatabase.Create(nil);
    sqlCur:=TSqlitePassDataset.Create(nil);

    sqlDB.Database:='ds_db.db3';
   
    
    -> NOT NECESSARY - DB3 EXTENSION FILE IS ENOUGH
   //sqlDB.DatabaseType:=dbtSqliteExpert;  // is this right / nessessary ? 
   

    sqlDB.Connected:=True;
    sqlCur.Database:=sqlDB;

    //quering
    sSQL:='SELECT a.*, b.WeCl2_Name, c.WeCl1_Name '+
               'FROM Weapons a '+
               'INNER JOIN WeaponClassification2 b ON a.WeCl2_ID=b.WeCl2_ID '+
               'INNER JOIN WeaponClassification1 c ON b.WeCl1_ID=c.WeCl1_ID '+
                'ORDER BY c.WeCl1_Name, b.WeCl2_Name, a.We_Name;';
    sqlCur.SQL.Text:=sSQL;
    sqlCur.Open;
   
    // no recordset-iteration, because of an easy example
    showmessage( sqlCur.FieldByName('WeCl2_Name').AsString );
   
-> NOT NECESSARY TO CLOSE SQLCUR AND SQLDB. THEY WILL BE CLOSED ANYWAY WHEN YOU FREE THEM.
    OR CLOSING SQLDB WILL CLOSE SQLCUR
   
   // sqlCur.Close;

    // closing and freeing objects. is this enogh?
    //sqlDB.Connected:=False;

    sqlCur.Free;
    sqlDB.Free;

Thanks

31

(5 replies, posted in Bug Tracking)

Kai,

Thanks for your Db sample. You were right. The TSqlitePassSQLTokenizer doesn't do his job properly...

The problem occurs in the function IsKeyword (located in SqlitePassSQLStmts.inc)

function IsKeyword: Boolean;
    var
    i: Integer;
    TempText: WideString;
    begin
    Result := False;
    { Checks if the token is a SQL Keyword }
    For i := 0 to High(SelectStmtKeywords) do
        begin
        SetString(TempText, TokenStart, Length(SelectStmtKeywords[i]));
        if AnsiCompareText(TempText, SelectStmtKeywords[i]) = 0 then
           begin
           Result := True;
           Inc(CurrentChar, Length(SelectStmtKeywords[i]));
           Break;
           end;
        end;
    end;

I fix this bug in the next 0.55 version in order to have a more efficient parser with quoted identifiers and reserved SQL words.

Let me know if you have already an idea on How to code this part.
Many thanks for your "assessment".

Regards.
Luc

32

(5 replies, posted in Bug Tracking)

Hello Kai,

I don't think the bug is due to the table name.
Did you create new tables or modify the existing ones with different tools (sqlite administrator,  sqlite expert ...) ?

Could you please send your db by mail to let me check this out ? (even with empty table or fake data  if you are concerned by privacy).

Thanks.
Luc

33

(3 replies, posted in General)

Hi Salsan,

To Execute an SQL, you just have to use a TSQLitePassDatabase (SqliteDb in the example), not a TSqlitePassDataset.
Here is an example:

...
SqliteDb.Engine.ExecSQL('CREATE TABLE `PROGRESSION` (FIELD1 INTEGER);');
SqliteDb.Engine.ExecSQL('BEGIN TRANSACTION; ALTER TABLE PROGRESSION ADD COLUMN FIELD2 TEXT(50); COMMIT;');
...

Regards

34

(2 replies, posted in General)

Hi,

Because those topics have been answered, but you still always open a new topic if you need to.

I couldn't answer the 64 bit Linux topic because I didn't have any 64 bit computer available around to test the components...but I've got one now...

Regards.

35

(4 replies, posted in General)

Hi Pascal,

The component always converts text fields to TWideStringField with d2009 and 2010.

The 30 chars default size is automatically set when no field size can be retrieved from the table create statement.

You should be able to change the default field type using the database.datatypeOptions.CustomFieldDefs but I didn't test it thoroughly yet...

Send your db by mail if you need to.

Regards.
Luc

36

(10 replies, posted in General)

Pascal,

Many thanks for your tests. The results are quite encouraging !

Database.options bugs are corrected in the upcoming 0.51 version. I added more pragma commands too.

Just waiting for new bugs now ... smile

I will post this new release soon.

Regards

37

(10 replies, posted in General)

May be for the 0.51 version, but this should be used with care with queries that involve many tables....

Could you tell me if your speed tests were ok (even if the components are not 100% reliable yet...)

Regards

38

(10 replies, posted in General)

Hi Pascal,

Lunchtime so I can answer your post smile

FRefetchRows  is used as a boolean flag when TSqlitePassetDataset.InternalRefresh needs to refetch records from disk.

but since I already override TDataset.Refresh, setting  FRefetchRows := True, it doesn't need to be set again in TSqlitePassDataset.EmptyTable.

Procedure TSqlitePassDataset.Refresh;
begin
  FRefetchRows := True;
  Inherited;
end;

{procedure InternalRefresh; virtual; abstract;
-------------------------------------------
Refresh the dataset.}
procedure TSqlitePassDataset.InternalRefresh;
begin
...
   {  Queries data using SQL sentence : Refetch the records from Sqlite database engine }
   Try
   if FRefetchRows then
      begin
      FRecordset.Open(FSQLSelectStmt.SQL, FSQLSelectStmt.FPrimaryKeyCount);
      MapFieldsToMemoryFieldBuffers;
      if Not FLookupFieldsDisplayedRecordsOnly
         then FillLookupFields(0, Pred(FRecordset.FRecordsCount));
      if Not FCalcDisplayedRecordsOnly
         then FillCalcFields(0, Pred(FRecordset.FRecordsCount));
      FRefetchRows := False;
      end;
...

Finally, I decided to change the behavior of EmptyTable :

When called directly from the current dataset, the data will be refreshed automatically

procedure TSqlitePassDataset.EmptyTable;
begin
  if DatasetType = dtTable then
     begin
     FDatabase.Engine.ExecSQL('DELETE FROM "' + DatasetName + '";');
     Refresh;
     end;
end;

Suggest : I could extend the syntax to include dtQuery ->  if DatasetType in [dtTable, dtQuery, dtSQLSelect] ?

When NOT called directly from the current dataset but from Database.TableDefs, the data will NOT be refreshed automatically (because you shouldn't be working on or displaying data from the table you want to empty...)

procedure TSqlitePassTableDefs.EmptyTable(Const TableName: String);
begin
 FDatabase.Engine.ExecSQL('DELETE FROM "' + TableName + '";');
end;

Regards

39

(4 replies, posted in General)

Hi Pascal,

The database.options are correctly only when the database is connected.

Obviously, I didn't finish the job for GetFSynchronous and GetFTemporaryStorage.

function TSqlitePassDatabaseOptions.GetFSynchronous: TSqlitePassSynchronous;
begin
  Result := FSynchronous;//TSqlitePassSynchronous(FDatabase.GetIntPragma('synchronous'));
end;

function TSqlitePassDatabaseOptions.GetFTemporaryStorage: TSqlitePassTempStore;
begin
  Result := FTemporaryStorage;//TSqlitePassTempStore(FDatabase.GetIntPragma('temp_store'));
end;

One more thing to do... smile

Regards

40

(10 replies, posted in General)

Hi Pascal.

Nothing else to do. The table data is deleted on disk directly after a call to TSqlitePassDataset.EmptyTable but the table is not automatically cleared from the dataset memory.

In the current implementation, you must call TSqlitePassDataset.refresh.

Does it make sense to you or do you think the refresh should be automatically applied ?

Regards.

Luc

41

(15 replies, posted in General)


Q1. Is dbtUnknown the correct type? Any other type is giving me a warning after the second time that I open the database: Error while loading data from database. The "SqlitePass_DbSettings" table is corrupted or was created using an older version of this component

You'll find the CreateDatabase prototype below. If you have a look at the code source (SqlitePassDatabase.inc) you'll notice that only "dbtKexi" is used in the current implementation. So any other type should produce the same result.


function TSqlitePassDatabase.CreateDatabase(DbName: String; DbType: TSqlitePassDatabaseType;
                                            DbEncoding: TSqlitePassEncoding = UTF8; PageSize: TSqlitePassPageSize = 4096;
                                            AutoVacuum: TSqlitePassAutoVacuumType = 0): Boolean;

I have to fix the other bug "Error while loading data from database. The "SqlitePass_DbSettings" table is corrupted or was created using an older version of this component".

Does it also occure when you open the db for the 3rd, 4th... time etc... ?
As a quick fix you can try to set the database.datatypeOptions.LoadOptions to soManual or the other database.datatypeOptions.LoadOptions properties to false.


At the moment I'm testing inserts and filters to see how fast the combination is.
- First test on a virtual xp machine, an average of 1000 records added in 8 seconds (8ms/record).

The filters are quite fast, especially if you use a "InMemoryIndex" = property Dataset.IndexedBy or if the dataset is already sorted by one of the filter field. (let me know how fast compared with other datasets (I didn't test)).

Concerning insert or edit operations your speed result seems to be very slow compared with my own tests except if your records are very big. Be sure to use a transaction first (SqlitePassDatabase1.Engine.Transaction.Start and commit or rollback).

Regards
Luc

42

(10 replies, posted in Bug Tracking)

Hi Pascal,

I didn't do too much testing with D2010 (lack of time).

I tried to follow your instructions but could't get the bug to occur.

Do you have special chars in your database path ?

Could you send you sample db by mail ?

Thanks.

43

(15 replies, posted in General)

Hi Pascal,

Thanks for trying the components.

I'm creating the database with:

dbSQLite.CreateDatabase('SQLitePassTest.db3', dbtUnknown, UTF16);

Q1. Is dbtUnknown the correct type? Any other type is giving me a warning after the second time that I open the database: Error while loading data from database. The "SqlitePass_DbSettings" table is corrupted or was created using an older version of this component.

This error only occurs when I create the databse with UTF16. It works fine with UTF8

I'm going to try your code, check this bug and  let you know soon.
By the way, you don't need to set the SQL statement for the TSqlitePassDataset component since it should be set automatically when you select a table or query via the datasetname property editor.


Q2. I'm using UTF16 because delphi 2010 uses this as default string type. Is this correct what I'm doing?

Suppose I create the database with UTF8 .. is this a problem because delphi 2010 strings are UTF16.

You are right, Delphi from 2009 uses UTF16 as default encoding for strings but it is not directly related to database encoding. An UTF8 or UTF16 database can be used with any Delphi or lazarus version.

Regards.
Luc

44

(1 replies, posted in Bug Tracking)

Hex,

Thanks for reporting this problem.

I currently prepare the 0.50 version. It should solve your problem(s) with lookup fields and bring a lot of speed improvement with in memory indexes for locate, lookup operations and lookupfields.

I plan to release the new package by the end of february.
As a quick fix, you can use SQL and join tables in a query in order to display the requested lookup information.

Regards.

Luc

45

(1 replies, posted in Bug Solved)

Hi,

You are right : This code is the default behavior for TList.

I voluntuary removed this check from the TSqlitePassWideStringList implementation to get a small speed gain.

It shouldn't be necessary to add this because the TSqlitePassWideStringList is directly managed by the TSqlitePassRecordset and shouldn't be used in another way.

Regards

46

(2 replies, posted in General)

Bonsoir Agur,

Après quelques essais, voici deux options pour solutionner ton problème.

Soit mettre le dataset en readonly := true avant d'exécuter la requête "select distinct..."
Soit changer le code suivant dans le fichier  SqlitePassSQLSmts.inc et recompiler la library.

Procedure TSqlitePassSelectStmt.AddPrimaryKeys;
  ....
  Tokenizer.Text := SQLSections.GetSectionText(kwSelect);
  Tokenizer.InsertAfter(kwSelect, FPrimaryKeyStmt);
  SQLSections.SetSectionTextFromTokenizer(kwSelect, Tokenizer);
  FPrimaryKeyCount := TablesName.Count;
  ...

par celui-ci

  Tokenizer.Text := SQLSections.GetSectionText(kwSelect);

  if Tokenizer.Locate(kwDistinct)
     then Tokenizer.InsertAfter(kwDistinct, FPrimaryKeyStmt)
     else Tokenizer.InsertAfter(kwSelect, FPrimaryKeyStmt);
  
  SQLSections.SetSectionTextFromTokenizer(kwSelect, Tokenizer);
  FPrimaryKeyCount := TablesName.Count;

La deuxième solution sera incluse dans la prochaine version. Elle évite le message d'erreur mais ne retourne pas une requête "Distinct"  réelle puisque une colonne rowid est ajoutée à la requête (pour écrire dans la base de données si besoin) et fausse le résultat. Donc, je te propose d'utiliser la 1ère solution.

@+

Luc

47

(2 replies, posted in General)

Hi Agur,

I don't have much information to find out what's going wrong with the query. SqliteToolbox is based on sqlitepass library with no add-on so if your query works inside SqliteToolbox it should also works with your unit.

You can check how SqliteToolbox handles your query by running the program inside the IDE and step through the code or you can send your code and db if your data or code are not confidential.

Regards.

Luc

48

(12 replies, posted in General)

Hi Agur,

Open the ..\SQLitePass\Sources\SqlitePassDbo.inc file and undefine the {$DEFINE HasftWideMemo} compiler directive.

    
{$IFDEF VER150}
      {$DEFINE Delphi7}
      {$DEFINE Delphi6}
      {.$DEFINE HasftWideMemo} // Support Unicode ftWideMemo
      {$DEFINE NeedSqlitePassWideStringUtils}
      {.$DEFINE UseTNTComponents}
    {$ENDIF}

It should fix the problem.

Let me know in order to update the next version.

Regards

49

(1 replies, posted in General)

Hi John,

I need one more clue to help you :

1 - Your table was created with sqliteadministator
2 - You edit your table with sqlitepass
3 - When you re-open the table and notice data is truncated, do you use sqliteadministrator or sqlitepass ?


if this occurs only when reading data back with sqlitepass, try to change the following code in SqlitePassDataset.inc

Function TSqlitePassDataset.GetFieldData(Field: TField; Buffer: Pointer): Boolean;
...

        ftString, ftFixedChar
                     : begin
                       StrValue := FRecordset.GetFieldAsAnsiString(ActiveRecIndex, FieldPos);
                       StrSize := Min(Length(StrValue)+1,Field.DataSize);
                       Move(PAnsiChar(StrValue)^, Buffer^, StrSize);
                       end;
  ...

with

        ftString, ftFixedChar
                     : begin
                       StrValue := FRecordset.GetFieldAsAnsiString(ActiveRecIndex, FieldPos);
                       StrCopy(Buffer, PAnsiChar(StrValue));
                       end;

if not, would you mind sending your complete code. I will check it out.
Unicode support in 0.45 version brought a lot of changes. I'm working on corrections for the next release.

Regards

50

(0 replies, posted in General)

Some of you had trouble installing SqlitePass package with Delphi 2010.

Actually,  I have some difficulties to get both Delphi 2009 and Delphi 2010 installed on the same machine.

(The Delphi 2010 trial version refuses to install and asks for a Delphi  2009 Update 3, althought update 4 is already installed ...)

As I don't have too much time to solve this right now, I would really appreciate if anyone could help me either to get Delphi 2010 installed or prepare a "Delphi 2010 SqlitePass package".

Thanks. smile