Results 1 to 10 of 12

Thread: Directory Listings

Hybrid View

  1. #1
    Join Date
    Feb 2008
    Location
    Scotland, UK
    Posts
    2,085

    Directory Listings

    Just wondering if anybody had any hints for retrieving lists of files and folders within directories.

    I know about DirectoryInfo and GetFileSystemInfos() and FileSystemInfo and so on, but the performance with them - particularly over a network - seems pretty dire, apparently due to them making too many network requests for each file returned, rather than storing the information that the underlying win32 calls they make has already received. It's something that's supposed to be much better with .NET4, but I guess we have to stay with .NET 2 for now? (and I'm not too sure how it works if you have a .NET2 compiled program but .NET4 is actually installed; can it use the latter or will it insist on running it with .NET2 anyway).

    I did find this (and a few like it), that sounded hopeful, but it only returns files within a folder, whereas I need lists of files *and* folders together.
    http://www.codeproject.com/KB/files/...numerator.aspx

    It's possible part of the problem is that I'm trying to be too smart when browsing a folder, as my plugin also goes looking into the subdirectories of the list returned (eg to determine if they might contain DVD rips). This means it's actually doing more than just a single GetFileSystemInfos() call, but I kind of need to know this information before building a UiList of possible videos to play. That way DVD folders and folders with just a single video in them can be presented at the same level as other video files found within a given directory.

    Edit: This was the other one I found: http://www.codeproject.com/KB/files/...px?msg=3177659

    Iain
    Last edited by imilne; 2011-08-20 at 07:05 PM.
    NPVR Plugins: AudioAdjust | AutoRefresh | MediaInfo | MovieTrailers | ReClockOrNot | SubMenu | System | Videos+

  2. #2
    Join Date
    May 2006
    Location
    Canada
    Posts
    20,482
    When I was doing the MB4 update to ML3 I tried a non-database mode, but the performance as you wrote wasn't up to it for real situations with large lists and networks. There is a one word solution. Database.

    Martin

  3. #3
    Join Date
    Feb 2008
    Location
    Scotland, UK
    Posts
    2,085
    Quote Originally Posted by mvallevand View Post
    When I was doing the MB4 update to ML3 I tried a non-database mode, but the performance as you wrote wasn't up to it for real situations with large lists and networks.
    How large was large in your case? I'm hitting problems with 200 folders, but that's 200 sets of file listings it has to do.

    Quote Originally Posted by mvallevand View Post
    There is a one word solution. Database.
    Yeah, I know. But I was *really* hoping to avoid going down that route. Shame it's not a one line of code solution too

    There's still a few things I can try before giving in... perhaps some fancy background threading along with priority updates for items on screen. That way the initial delay would only be for a single file listing, although it would compromise the ability to do a few things, such as different sort orders for items based on anything read from the metadata, because you wouldn't have all the information available at the time a list was built.

    Iain
    NPVR Plugins: AudioAdjust | AutoRefresh | MediaInfo | MovieTrailers | ReClockOrNot | SubMenu | System | Videos+

  4. #4
    Join Date
    Feb 2008
    Location
    Scotland, UK
    Posts
    2,085
    Quote Originally Posted by imilne View Post
    There's still a few things I can try before giving in... perhaps some fancy background threading along with priority updates for items on screen.
    Ignoring the rest of my waffling for the moment, is what I've said here possible, ie, can a plugin determine which UiList items are the ones on screen (and therefore need their details fetched from disk *now*). A trick with image callbacks perhaps? Because the renderer only requests images for currently visible items does it not?

    Or perhaps a method could be added to UiList.IUiListCallback, eg, ItemsVisible(UiList.ListObject[]).

    Iain
    Last edited by imilne; 2011-08-21 at 02:55 PM.
    NPVR Plugins: AudioAdjust | AutoRefresh | MediaInfo | MovieTrailers | ReClockOrNot | SubMenu | System | Videos+

  5. #5
    Join Date
    Nov 2003
    Location
    NextPVR HQ, Wellington, New Zealand
    Posts
    75,527
    Quote Originally Posted by imilne View Post
    Ignoring the rest of my waffling for the moment, is what I've said here possible, ie, can a plugin determine which UiList items are the ones on screen (and therefore need their details fetched from disk *now*). A trick with image callbacks perhaps? Because the renderer only requests images for currently visible items does it not?
    Yes, you can implement the SkinHelper.IUpdateOnRenderElement interface, and add a "@callbackOnRender" = this parameter to your element. It'll give you an opportunity to do something to a list item before it is displayed.

  6. #6
    Join Date
    Feb 2008
    Location
    Scotland, UK
    Posts
    2,085
    Quote Originally Posted by sub View Post
    Yes, you can implement the SkinHelper.IUpdateOnRenderElement interface, and add a "@callbackOnRender" = this parameter to your element. It'll give you an opportunity to do something to a list item before it is displayed.
    Oooh, sounds perfect. Thanks.

    Iain
    NPVR Plugins: AudioAdjust | AutoRefresh | MediaInfo | MovieTrailers | ReClockOrNot | SubMenu | System | Videos+

  7. #7
    Join Date
    Sep 2005
    Location
    Lower Hutt, NZ
    Posts
    4,872
    Quote Originally Posted by imilne View Post
    Oooh, sounds perfect. Thanks.

    Iain
    Did you get this to work? So does this mean if you select an element it gets the image?

    Is there a way of getting a bunch of images slow time (another thread maybe) and update the UiList later. So the first images may not be there, but theyll turn up later.
    AMD FM1 x4 A8-3870 Llano, Gigabyte GA-A75-D3H, 4 Gb DDR3 Ram, Seagate 500 Gb, Seagate 1500 Gb, HVR3000 x 2 (DVB-S), Nova-T 500 (DVB-T), USB-Uirt, SPDIF -> Yamaha RX-V540, ATI HD6550 (OnBoard) > Samsung LA46A650 TV (HDMI), Harmony Remote 525
    Windows 7 Ultimate x64 SP1, NPVR 2.4.3, AC3Filter, 1 x Popcorn Hour A-100, 1 x AppleTV with Plex

  8. #8
    Join Date
    Feb 2008
    Location
    Scotland, UK
    Posts
    2,085
    Hmm, a folder with about 1000 files (or subfolders) accessed over Wireless-N seems to take about 300-400ms to return its listings. That's still pretty crap. Gigabit wired takes about 1/10 of that.

    Iain
    NPVR Plugins: AudioAdjust | AutoRefresh | MediaInfo | MovieTrailers | ReClockOrNot | SubMenu | System | Videos+

  9. #9
    Join Date
    Feb 2008
    Location
    Scotland, UK
    Posts
    2,085
    Some more observations:

    If you get a listing with GetFileSystemInfos() (which returns both files and directories), then it takes 300-400ms to do that. But you still don't know which are files and which are directories until you look at the FileAttributes. That (stupidly) seems to make *another* network request for each item, taking the total time up to around 8 seconds!

    However, if you get the list in two separate calls: one to GetFiles() and another to GetDirectories(), then the time drops back down to 300-400ms, I guess because the remote system is doing the type-determination before returning anything, rather than having to do it over the network item by item..

    Iain
    NPVR Plugins: AudioAdjust | AutoRefresh | MediaInfo | MovieTrailers | ReClockOrNot | SubMenu | System | Videos+

  10. #10
    Join Date
    Mar 2007
    Location
    Hartland, VT USA
    Posts
    156
    Quote Originally Posted by imilne View Post
    Some more observations:
    But you still don't know which are files and which are directories until you look at the FileAttributes. That (stupidly) seems to make *another* network request
    Iain
    It's been a while since I've tried something like this, but what about PINVOKE-ing FindFirstFile/FindNextFile and creating a List<WIN32_FIND_DATA>. Once you have them in memory, you can step through them at your leisure and not have to worry about further network hits. The dwFileAttributes == FILE_ATTRIBUTE_DIRECTORY should be the key.
    Let me state I'm thinking off hand at this point, and haven't tried any code to see if performance will be better.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •