Performance problems (yes)

Sorry to come back again with this! And sorry to bother you with that after your hard drive crash, Pierre... but it's becoming very difficult for me to work :
IQ, even if the greatest PIM in the world, really slows me down these days.
 
Quite possible that users not using equations/autoassigns/conditional formatting don't experience those performance problem.
 
------
 
I don't know if the way equations/auto-assigns are managed has changed in the last few months (?), or if it's related to the new grid "refreshing" routines/events, but each time some change is made to an item -- and especially if there are some equations/auto-assigns involved, it's really, really slow. And by slow, I mean... s-l-o-w.  
 
5-8s waiting here and there disrupts my thinking process. I could post a screen capture if necessary.
 
I tried working with only one grid, but it's not practical and the difference isn't that great.  I'm not trying to send a spaceship to Mars here, I'm just marking items as done, adding dates through auto-assign equations and stuff like that!   
 
 
Anybody else experiences slowness in specific tasks ? Descriptions could help isolating the problem(s).
 
Tests would have to be made with a relatively large DB (to be somewhat realistic...), a few open grids, fields with auto-assign equations + row/column equations, and conditional formatting.
 
I'd like to give more details, but I'm in the middle of moving + writing a thesis, so it makes it real hard. (Note that there are various threads in the forum where I detailed performance problems regarding auto assigns/equations)
 

Re: Performance problems (yes)

Hi Armando,
 
Sorry to hear you are having performance issues. I have a 12 year old notebook PC (yes, I know, an antique it is), years of stuff in it and have no performance issues...
If you can give me the last version which did not have that issue, I'll be able to see what has changed.
 
p.s. You and I know that you have an "extremely" complex database and that what you now do with IQ, you could not do with any other piece of software...

Re: Performance problems (yes)

Just spent 45 min timing and writing stuff, and I lost my post hitting backspace. The web page reverted to the previous page... More time down the drain.
Yes, as I said IQ is the best PIM, in terms of flexibility, etc. But it doesn't mean that performance can't be improved.
 
If I post these concerns, it's not to bash IQ, but because I hope that some user will recognize something and will be able to contribute some insight. 
 
Anyway, this should give an idea.
 
This is in a Boolean field called FollowUp
 
auto-assign:
 
A:FollowUpDate=followupValue(TaskActStart, FollowUp, FollowupDate, taskStartDurLock, Rechour) | 
 
 
The Function it refers to is:
 
Function followupValue(TaskActStart, FollowUp, FollowUpDate, locked, recHour)
  Dim rightHour
  If (Not IsNull(recHour) Or  recHour > 0)  Then rightHour = recHour Else rightHour = 0.25
  If followUp = False Then
    followupValue = Null
    Exit Function
  ElseIf locked = True And Not IsNull(followupdate) Then
    followupValue = FollowUpDate
    Exit Function
  ElseIf (((TaskActStart >= Now) Or (rightHour >= Now-Int(Now))) And (Int(TaskActStart) - Int(Now) <= 0)) Or (IsNull(TaskActStart) And rightHour >= Now-Int(Now)) Then
    followupValue = Int(Now) + (rightHour)
  ElseIf (TaskActStart < Now Or (rightHour < Now-Int(Now))) Or (Int(TaskActStart) - Int(Now) > 0) Or (IsNull(TaskActStart) And rightHour < Now-Int(Now)) Then
    followupValue = Int(Now) + 1 + (rightHour)
  Else
    followupValue = FollowUpDate
  End If
End Function
 
Yes, it's ugly as hell. But, to my defense, it was written 7 years ago and it does what I want. Not sure if it could be tweaked so that it runs quicker. Maybe, but I don't think I'd gain a 50% performance increase from that work. Maybe...
 
Anyway just applying that auto-assign equation to 100 items takes 43s
 
Now add those auto assigns (as in my own DB):
 
A:itemforecolor="&h91CFFF" |
A:icon_statut=AddKeyword(icon_statut, "<img>hourglass_trans</img>") | 
 
And the total is now : 64s (this takes into account the "refreshing", of course as it's not possible to do anything while IQ is refreshing the grid cells)
 
For 10 items it's 6.5 s.
 
IMO, it's slow. 
 
Let's take a simpler case
 
fictive field : addNumToNum
 
auto-assign: 
 
A:order = order + 3
 
Applying  this to 10 items takes 2.4
Applying  this to 50 items takes 10.5s
Applying this to 100 items takes 20.9s
 
It's still not super quick.
 
those are relatively simple examples as there are no row or column equations involved at the same time. Add rows and columns into the mix [Edit : actually, in my first example, there is a row equation involved in another field, and it doesn't help, of course] and...
 
 
[EDIT : I just want to make sure everybody understands that I do know that my equations etc. aren't super simple. But, at the same time, I'm wondering if  21 sec. to add "3" to 100 items is a normal speed.  Is it ?  If it is, then I'll just accept the situation, swallow the pill, and simplify all my DBs equations -- which I should do anyway.]

Thanks all.
 

-------------------------------------------------------
Windows 8.1
Sony Vaio S Series 13 (SVS131E21L)
Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

Re: Performance problems (yes)

OK, I've tested it on a sample database.
On my "slow" PC the times are (for 100 items)
  1. FollowUp auto-assign 8 seconds (you have 64)
  2. addNumToNum: 4 seconds (you have 20.9)
I've enclosed the sample IQBase (using grid Inbox). Can you check what are your times with this file ?
 
[edit] Approx same times with my new Windows 8.1 tablet, an Acer Switch 10.
Not stellar, but the fan-less quad-core processor run at around 40C and consumes only 0.25W on "idle" and 0.75W when IQ is working 100%... amazing !!!
 
AttachmentSize
test.zip298.87 KB

Re: Performance problems (yes)

Thanks a lot for looking into that!
 
I'm only going to look/work at/with these numbers as they aren't affected by any row/column, equations :
 
fictive field : addNumToNum
 
auto-assign:  
A:order = order + 3
 
Applying this to 100 items takes 20.9s
 
This is 5 times the times you get. I'm puzzled, but I don't have the time to explore more unfortunately  (moving, urgent deadlines...)
Maybe it has to do with the number of columns ? The size of DB ? the number of grids ? the number of fields ? So many parameters.
 
-------------------------------------------------------
Windows 8.1
Sony Vaio S Series 13 (SVS131E21L)
Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

Re: Performance problems (yes)

 Try it on the sample app, we'll be able to have a better idea of the cause
 

Re: Performance problems (yes)

addNumToNum
 auto-assign:  
A:order = order + 3
 
Applying this to 100 items :
 
 
On my DB, 1 grid with 4 columns : 8.6 sec
 
On my DB, 1 grid with 44 columns : 8.7 sec
 
with 5 other opened grids : 12.1
 
with 10 grids : 21.8
 
with 20 grids : 23.4
 
 
So, preliminary conclusions:
 
1- my DB *is 4x slower* (maybe because it's bigger? Number of fields? number of items?). Not that addNumToNum is an autonomous field, and that the  order field is not linked to any row/column equations in other fields. However, there's some conditional formatting involved. Conditional formatting might affect performance -- will test later.
2- adding more grids to the mix seems to really affect performance. However, the difference between 1, 5 and 10 is big, but not between 10 and 20.
 
I don't have the time to do more tests right now.
 
I *never* work with less than 10 opened grids. The norm is actually closer to 15-20.
Simply because there are no ways to quickly classify, prioritize  and open groups of grids in IQ.
Depending on what you're doing (not if your only managing a phone list, a task list, personal finances and a few simple projects... ) it's way easier to have your work right there than buried in 100s of other grids stacked in a list. So the only solution is to open them and put them in a separate tab group (& maybe "hide" the tab group once in a while, etc.).
 
 
 [2015 02 06 -- 12 21 : edited post to add details]
-------------------------------------------------------
Windows 8.1
Sony Vaio S Series 13 (SVS131E21L)
Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

Re: Performance problems (yes)

In v0.9.26PreRel40, I'll only update the current grid. This should speed things up significantly. 
We'll see if it is penalizing in any way to do thing this way...
(quite a few apps work this way... to update a window, one must give it the focus)
 

Re: Performance problems (yes)

Thanks Pierre. You answered as I was was writing my post (below)
 
----
 
Grrrr... Wondering why the numbers were so different from the sample, I found a "hidden" equation indirectly linked to "order". Removed it, presto.
 
Now the numbers are still a *bit* bigger, but not by that much; so the problem doesn't lie with the size of my DB (I'm relieved!!!!) nor with the amount of fields/grids (relieved again).
 
New numbers with the deleted extra equation
 
with 20 grids, 100 items : 14.7
with 10 grids, 100 items : 11
with 1 grids 100 items :5.1
 
removed conditional format totals:
 
with 20 grids, 100 items : 14.7
with 10 grids, 100 items : 10.2
with 1 grids 100 items : 5
 
 
OK. So I'm basically back to square one : the number of opened grids + row equations/auto-assign equations are what slows down everything the most.
 
-------------------------------------------------------
Windows 8.1
Sony Vaio S Series 13 (SVS131E21L)
Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

Re: Performance problems (yes)

This is good news !
What do you think of my idea of only updating a grid once it receives the focus ?
Better still, I'll try to put the update in another thread, and adjust the update frequency based on whether the grid has focus or not.
Basically, non-focused grids will get updated, but at a slower rate
 

Re: Performance problems (yes)

That approach seems good. The only potential problem is the background updating of other grids.
 
I wonder if IQ could do the updating ether both
 
- as you suggested, when the grid receives the focus
- as you suggested, in another thread, but only when IQ/the PC isn't in use; this would accelerate performance all around. Maybe it's not feasible though.
 
Updating the non-focused grids at a constant (lower) rate might work, but I'm not sure how much will this still affect IQ's operations as it's trying to keep up with all the queued changes.
 
 
[Edited my meaningless post]
 
-------------------------------------------------------
Windows 8.1
Sony Vaio S Series 13 (SVS131E21L)
Ram:8gb, CPU: Intel i5-3230M, 2.6ghz

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.