Opened 11 years ago

Closed 9 years ago

Last modified 9 years ago

#461 closed defect (worksforme)

loading large CMDI results in high server and client load

Reported by: dietuyt Owned by: Twan Goosen
Priority: minor Milestone:
Component: VLO web app Version:
Keywords: Cc: teckart

Description

When loading a 8.8 MB CMDI file my browser (most recent firefox and chrome on Mac) becomes very slow, for a very long time (half an hour or so). At the same time the CPU load on catalog.clarin.eu goes to 100%.

Of course this is an exceptionally big file, but the VLO should be able to deal with it in a more robust way.

Change History (11)

comment:1 Changed 11 years ago by DefaultCC Plugin

Cc: teckart added

comment:2 Changed 11 years ago by twagoo

In 2.18 (as of r4249) this is somewhat redeemed because the transformation only takes place on request (when the user clicks 'Show CMDI metadata').

No structural fix, so leaving ticket open.

comment:3 Changed 10 years ago by Twan Goosen

Related to #520

comment:4 Changed 10 years ago by Twan Goosen

Owner: changed from keeloo to Twan Goosen
Status: newaccepted

comment:5 Changed 10 years ago by Twan Goosen

Component: VLO web appVLO importer
Milestone: VLO-3.1

For 3.1 let's experiment with higher thresholds. Since the transformation is carried out lazily, should not be such a problem for the web app anymore. It would be nice to benchmark the performance of the transformation process in isolation first (using a range of file sizes) to get a rough idea of the impact.

Moving this to the importer, because the importer decides which records to skip.

comment:6 Changed 10 years ago by Twan Goosen

Component: VLO importerVLO web app

The maximum file size accepted by the importer is governed by the 'maxFileSize' parameter in
VloConfig.xml. Experiment with this and investigate the behaviour of the web app with files this large.

comment:7 Changed 10 years ago by Twan Goosen

Priority: criticalminor

comment:8 Changed 9 years ago by Twan Goosen

Milestone: VLO-3.1VLO-3.2

Splitting 3.1 milestone. Most open tickets go to 3.2 so that we can have a release on the short term.

comment:9 Changed 9 years ago by Twan Goosen

The server seems to cope quite will with the mentioned example record, with load staying at normal levels.

User experience is not so good, because asynchronous requests for each resource (around 4000 in this case) make the other Ajax links on the page unresponsive for quite a long while. Paging should resolve that, see #565. Closing this ticket.

comment:10 Changed 9 years ago by Twan Goosen

Resolution: worksforme
Status: acceptedclosed

comment:11 Changed 9 years ago by Twan Goosen

Milestone: VLO-3.2

Removing invalid/wontfix/worksforme tickets from 3.2 milestone

Note: See TracTickets for help on using tickets.