Paginated item list window

[Tsukihime]

I am looking for a script that changes the way items are drawn in a window.


Consider the item window. Imagine you have a million items in your inventory. The engine first grabs all of the items, creates a bitmap large enough to hold all of them, and then draws every single item.


This is not an issue with the default engine, because the database limits are done in such a way that you cannot draw more than 1000 items in a window. Introduce some database limit breakers or instance items, and suddenly you may be drawing 10000 items easily.


The solution is quite simple: why are we drawing things we don't need to see?


The default window only shows 22 items; it is not necessary to draw a million other items that you will likely never see.


What I am looking for is a smarter item list: it will only draw a very small subset of items, for example 40, instead of a million.


The bitmap will be redrawn as necessary if the player is trying to view items outside of the current range. For example, suppose you have 100 items in your inventory, and you have just opened up the item window and see the first 40 items in two columns. Now, the player starts scrolling down, and once they reach the last row and want to keep scrolling down, now you need to shift your offset from 0 to 2, because instead of showing items 0 to 40, you now need to show items 0 to 42.


This will greatly improve all item-drawing performance, and allow you to carry a million items without the game crashing on you because it couldn't draw a bitmap that was a 32 million pixels in height.


It can be written for any RM. Preferably Ace, since then I can just use it, but otherwise I don't mind porting it over since this is an essential script.

 

[Engr. Adiktuzmiko]

it just seems like overloading the draw_all_items method + the page scroll methods

you modify the scroll so that you can save the "offset" and refresh the window, and the draw_all_items method to only draw a specific amount...

class Window_ItemList  attr_accessor :offset    alias initialize_offsets_items initialize  def initialize(x, y, width, height)    initialize_offsets_items(x, y, width, height)    @offset = 0  end  def draw_all_items    (@offset..(@offset+21)).each {|i| draw_item(i) }  end  def cursor_pagedown    if top_row + page_row_max < row_max      self.top_row += page_row_max      select([@index + page_item_max, item_max - 1].min)      @offset += col_max      refresh    end  end    def cursor_pageup    if top_row > 0      self.top_row -= page_row_max      select([@index - page_item_max, 0].max)      @offset -= col_max      refresh    end  endend

This MIGHT work... can't test it... this should draw only 22 items at a time, and refresh/redraw when you move to another "page"... and well, this is only for Item List... but you can simply change the class to Window_Selectable I think to affect all selectable windows, and maybe modify the initialize call to adjust... though you'd need to overload those draw_all_items in case each list will show a different MAX number of items

 

[Selchar]

Nope, not quite the desired effect.  It draws the first 22 items, and when go look at the 23+ item, you see that they are not "drawn" or updated to be drawn, but still there.  I believe I found a script made by Mithran for VX which may have the desired effect, tho it of course doesn't work for Ace.

http://forums.rpgmakerweb.com/index.php?/topic/1070-window-selectable-max/

 

[Engr. Adiktuzmiko]

oh... because page up/down are only run for L/R buttons...

EDIT: w8... I think it will still error cause the bitmap is still being made for the whole...

so you actually need to just draw a bitmap as big as the window, then also modify the item rects so they will be based on index-offset rather than just index

EDIT 2: this should do it... draws the bitmap to only the window size...

module Offset  ITEMS = 21endclass Window_ItemList  attr_accessor :offset    alias initialize_offsets_items initialize  def initialize(x, y, width, height)    initialize_offsets_items(x, y, width, height)    @offset = 0  end    def item_rect(index)    index -= (@offset)    rect = Rect.new    rect.width = item_width    rect.height = item_height    rect.x = index % col_max * (item_width + spacing)    rect.y = index / col_max * item_height    rect  end   def draw_all_items    (@offset..(@offset+Offset::ITEMS)).each {|i| draw_item(i) }  end  def cursor_pagedown    if top_row + page_row_max < row_max      self.top_row += page_row_max      select([@index + page_item_max, item_max - 1].min)      @offset += col_max      refresh    end  end    def contents_height    return height - standard_padding * 2  end  def cursor_pageup    if top_row > 0      self.top_row -= page_row_max      select([@index - page_item_max, 0].max)      @offset -= col_max      refresh    end  end  def cursor_down(wrap = false)    if index < item_max - col_max || (wrap && col_max == 1)      x = (index + col_max) % item_max      select(x)      if x - @offset > Offset::ITEMS+1        refresh      end    end  end   def cursor_up(wrap = false)    if index >= col_max || (wrap && col_max == 1)      x = index - col_max + item_max) % item_max      select()      if x - @offset > Offset::ITEMS+1        refresh      end    end  end    def cursor_right(wrap = false)    if col_max >= 2 && (index < item_max - 1 || (wrap && horizontal?))      x = (index + 1) % item_max      select(x)      if x - @offset > Offset::ITEMS+1        refresh      end    end  end    def cursor_left(wrap = false)    if col_max >= 2 && (index > 0 || (wrap && horizontal?))      x = (index - 1 + item_max) % item_max      select()      if x - @offset > Offset::ITEMS+1        refresh      end    end  endend

 

[Tsukihime]

That one loads up my inventory of a million items in 1 ms, but it doesn't seem to be drawing the items properly once you start scrolling down.


There are a couple issues with resetting the cursor (eg: you hit cancel, then choose the same category again).


It's a good start though.


I'll see if Mithran's does what I need as well.

 

[Engr. Adiktuzmiko]

what happens when you scroll down? for selecting... hmmm... maybe the refresh should be placed in select instead... I'll just continue it at home later... and you can also just port Mithran's... but I'll finish mine just for the sake of making it...

 

[Tsukihime]

I looked at mithran's script but it's something completely different.


I think it could possibly address having to load a million items, but the focus of this script is to NOT load a million items.


It takes half a second for me to load 500 items. A million items is going to crash. Or you'll wait an hour.


It's just unscalable and terrible. We might as well replace Window_Selectable completely to use paginated fetching since there is no performance loss if you're only drawing 20-40 items.


No realistic system is going to ever fetch all records and then try to display all of them.

 

[Engr. Adiktuzmiko]

I see... I'll work on this again once I get home...

 

[Mithran]

Yeah, the focus of that script was to address the issue in VX where creating a bitmap from scratch was capped at an absurdly low amount (eg., not even enough to draw every item in a standard database). The script is built to draw the current window, then start drawing 'ahead' of the current position at a rate of I believe 1 draw/frame so scrolling can be sped up without having to wait for the whole page to draw. All bitmaps for the entire draw area are created at initialization of window, however, so there is that, that is probably the delay you are experiencing. It was designed this way for maximum compatibility and because drawing 20-40 items at a time was actually causing a noticeable delay (in my demo, if I pass my draw ahead point, my framerate is halved while scrolling). The design could be made to scale by using only a single buffer bitmap to draw the items on (to a certain point before/after the cursor position) and blt them over when scrolling, but by that point, yeah, you are probably better off just drawing a page at a time. Having draw_text be the bottleneck is probably eliminated now with ACE + my Text Cache.

 

[Engr. Adiktuzmiko]

Here... it works fine for me...

Spoiler

module Offset  ITEMS = 13endclass Window_ItemList  attr_accessor :offset    alias initialize_offsets_items initialize  def initialize(x, y, width, height)    initialize_offsets_items(x, y, width, height)    @offset = 0  end    def top_row=(row)    row = 0 if row < 0    row = row_max - 1 if row > row_max - 1    #self.oy = row * item_height => so that the bitmap won't scroll  end    def item_rect(index)    index_n = index - @offset #since the bitmap doesn't scroll anymore, need to adjust    rect = Rect.new    rect.width = item_width    rect.height = item_height    rect.x = index_n % col_max * (item_width + spacing)    rect.y = index_n / col_max * item_height    rect  end    def select(index)    if index - @offset < 0 #to avoid errors when cancelling item selection      @offset = 0    end    self.index = index if index    refresh #needed to ensure the cursor stays right when scrolling  end    def contents_height    return height - standard_padding * 2 #to keep the bitmap to the size of the window  end   def draw_all_items    @offset = 0 if @offset.nil? #anti-fail for saved games    if @offset+Offset::ITEMS <= item_max - 1      (@offset..(@offset+Offset::ITEMS)).each {|i| draw_item(i) }    else      (@offset..(item_max-1)).each {|i| draw_item(i) }    end  end    def cursor_pagedown    if @index + page_item_max >= item_max - 1      @offset = item_max-1      select(item_max - 1)    else      @offset += page_item_max      @offset = [@offset,item_max-1].min      select(@index + page_item_max)    end  end    def cursor_pageup    if @index - page_item_max < 0      @offset = 0      select(0)    else      @offset -= page_item_max      @offset = [@offset,0].max      select(@index - page_item_max)    end  end  def cursor_down(wrap = false)    if index < item_max - col_max || (wrap && col_max == 1)      x = (index + col_max)      if x - @offset > Offset::ITEMS        @offset += col_max      end      select(x)    end  end   def cursor_up(wrap = false)    if index >= col_max || (wrap && col_max == 1)      x = (index - col_max)      if x - @offset <= 0 and not @offset == 0        @offset -= col_max      end      select(x)    end  end    def cursor_right(wrap = false)    if col_max >= 2 && (index < item_max - 1 || (wrap && horizontal?))      x = (index + 1)      if x - @offset > Offset::ITEMS        @offset += col_max      end      select(x)    end  end    def cursor_left(wrap = false)    if col_max >= 2 && (index > 0 || (wrap && horizontal?))      x = (index - 1)      if x - @offset <= 0 and not @offset == 0        @offset -= col_max      end      select(x)    end  endend

 

[Selchar]

Edit: Was counting wrong, had 99 times more numbers, new numbers should be accurate.


Default Behavior:


Crashes somewhere between 129591 and 131274 items


Lags like heck on item window opening at 129591 and before


2-3 second lag at 26928 items


Seamless movement between items after loading regardless of item count(if no crash)


With Shana's Code


No crashing at 168300 items


Improved load time when opening item window.


Slight lag moving between items at 43758 items.


1 second lag at item window opening around 57222 items.


Lag at movement between items more noticeable at 85833 items.


2.5 second lag at item window opening, and 2 second lag at movement between items with 127908


5 second lag at item window opening, and 3 second lag at movement between items with 168300 items.


All tests done with Tsukihime's Instance Items, with Items set to Instance Mode, Battle Test and subtle changes to the amount of items gained through setup_battle_test_items method.


I'd say it's an overall improvement, and there's still room to improve. I haven't looked too closely at the code to see what all can be done, but it's a step in the right direction to see what's possible.


Edit: No need for too much improvement, seems to work just fine now that I rechecked my numbers.

 

[Engr. Adiktuzmiko]

maybe due to the large list itself (maybe accessing data in higher indices is inherently slower?)... You can probably try tweaking it to further improve it...


the default won't have lags at high numbers if it doesn't crash since it's just moving the bitmap when scrolling... unlike with this that we need to redraw things when you scroll since the bitmap is now fixed... anyway, I guess that's the trade-off...

 

[Tsukihime]

With Shana's Code


No crashing at 1700 items


Improved load time when opening item window.


Slight lag moving between items at 442 items.


1 second lag at item window opening around 578 items.


Lag at movement between items more noticeable at 867 items.


2.5 second lag at item window opening, and 2 second lag at movement between items with 1292


5 second lag at item window opening, and 3 second lag at movement between items with 1700 items.


All tests done with Tsukihime's Instance Items, with Items set to Instance Mode, Battle Test and subtle changes to the amount of items gained through setup_battle_test_items method.


I'd say it's an overall improvement, and there's still room to improve. I haven't looked too closely at the code to see what all can be done, but it's a step in the right direction to see what's possible.

Unless he's changed the code or something, I can load 100k item inventory in less than 10ms and scroll through it no problem.


I'm referring to the normal item scene, not the battle items. There seems to be something different with the battle item window.

 

[Selchar]

Could just be my laptop then, it is about 6 years old and has been through a lot. or maybe I'm not counting the items in my inventory correctly and it's actually more than I'm reading. I'll try some other tests.

 

[Engr. Adiktuzmiko]

I posted a new code before Selchar's post... which fixed the previous problems of items not drawing and weird behavior when cancelling etc... that might have had an effect or something...

 

[Tsukihime]

Could just be my laptop then, it is about 6 years old and has been through a lot. or maybe I'm not counting the items in my inventory correctly and it's actually more than I'm reading. I'll try some other tests.

Nvm, the battle item window does lag a lot for some reason, which...I'm not sure why.

 

[Engr. Adiktuzmiko]

maybe battle items is doing a bit more...

 

[Selchar]

Recopied the code, and redid the method that gave me items double checking to make sure it counts correct, turns out I had 168300 items instead of 1700. I'll edit my above post with proper numbers once I have them.

 

[Engr. Adiktuzmiko]

that's because BT gives 99 of each item, and you set them to instanced using Hime's script... so it became 99*1700

 

[Selchar]

Yup, sounds about right, my first time testing I forgot to expand that 1 line if statement and count within it, and keep results consistant(multiplying by 99 or whatever I was changing it to.)