1000 records are batched into each chunk and posted to event queue. Every chuck needs to be displayed on a “main” table. Killer is the re-sort/re-filter, which must happen on EDT. As a result, EDT is overwhelmed — when you re-size or move the app, it responds, but is sluggish and leaves an ugly trail.
My suggestions –
* Break up into smaller tables, so sorting is faster.
* Break up into smaller datasets. In fact most users are only interested in a subset of the live data.
* If sort (filter) is the killer….
… adopt a a sort-free JTable that efficiently accesses pre-sorted underlying data. Underlying data can be swapped in/out efficiently, and sorted on a “mirror-site” using another thread, possibly a swing worker. Take inspiration from the 2 GC survivor spaces and use 2 identical data structures. One live, the other used as a sort worktable. Swap (by pointer) and fire table event. However this increase memory consumption.
We also need to completely disable user-triggered sorts. Use a dialog box to educate users that changing sorting policy is non-trivial, and they should configure and install the new sorting policy and be prepared to wait a while to see its effect.
In general, you can only get any 2 of
E) cell editing
R) real time view of fast changing mkt data
C) full rather than subset of data streams
In mkt data streaming, R alone is often enough — read-only view. Some users needs R+C. Some needs E+R