Page 1 of 22 12311 ... LastLast
Results 1 to 10 of 215

Thread: Free XMLTV EPG scraper

  1. #1
    Join Date
    Mar 2005
    Location
    Nova Scotia
    Posts
    227

    Free XMLTV EPG scraper

    In previous threads alternatives to the zap2it TV guide have been mentioned.

    One very usable and easy to setup alternative is available at:

    http://planetreplay.com/phpBB2/viewtopic.php?t=14314

    I thought I would start a separate thread for this free XMLTV EPG scraper, as I believe a few other GBPVR users have tried this little utility. Some additional work is required to get it to work better with GBPVR.

    First a few good points -
    - It is free
    - Does not required MCE or VISTA
    - Supports Canadian users - including OTA & ExpressVu & Starchoice

    Now a problem I am having

    I can download program listings through the utility, and set the EPG source in config.
    But in the GBPVR TV guide I can see all the channels - but the message (no show details) are displayed for every time slot. But I do see program/TV show details in the generated XML file.

    After talking with drlava, it may be that gbpvr dosen't read in more than one <display-name> tag per channel. Or where GBPVR gets the channel number from - the ID, or the first <display-name> tag.

    So there are a few questions that I am sure others here will be interested in helping out with.
    Silent AOpen I855GMe-LFS, Pentium M 1.5GHz, SPDIF digital audio output, and onboard Intel Extreme Graphics 2 - HD-15 Analog Output, 1GB RAM, 2x 250GB SATA in RAID config, 160GB IDE with XP Pro SP2, ND-2500 DVD, 1x PVR-150, Home Theater Master MX-500 Remote Control, Sansui HDPDP4200 Plasma @ 1280*720, 2x MVP, CAV-T04-UKC case, APC BX1200-CN UPS, GB-PVR: 98.8

  2. #2
    Join Date
    Nov 2006
    Location
    Louisville, KY, USA
    Posts
    7,719
    Quote Originally Posted by dneprrider View Post
    In previous threads alternatives to the zap2it TV guide have been mentioned.

    One very usable and easy to setup alternative is available at:

    http://planetreplay.com/phpBB2/viewtopic.php?t=14314

    I thought I would start a separate thread for this free XMLTV EPG scraper, as I believe a few other GBPVR users have tried this little utility. Some additional work is required to get it to work better with GBPVR.

    First a few good points -
    - It is free
    - Does not required MCE or VISTA
    - Supports Canadian users - including OTA & ExpressVu & Starchoice

    Now a problem I am having

    I can download program listings through the utility, and set the EPG source in config.
    But in the GBPVR TV guide I can see all the channels - but the message (no show details) are displayed for every time slot. But I do see program/TV show details in the generated XML file.

    After talking with drlava, it may be that gbpvr dosen't read in more than one <display-name> tag per channel. Or where GBPVR gets the channel number from - the ID, or the first <display-name> tag.

    So there are a few questions that I am sure others here will be interested in helping out with.
    GB-PVR will only read the first display-name attribute. The channel= attribute in the <programme> tag should match the id= attribute in the <channel> tag. The display-name should have no effect on the programs.

    Code:
      <channel id="001">
        <display-name>LOOR001</display-name>
      </channel>
      <programme start="20070802020000 +0000" stop="20070802060000 +0000" channel="001">
        <title>Local Origination</title>
      </programme>
    It sounds as if the output of that program is not truly xmltv compliant. Do you have an output file you can post? OTA would be great because it shouldn't be too big.

  3. #3
    Join Date
    Nov 2006
    Location
    Louisville, KY, USA
    Posts
    7,719
    I tested the program and it looks like the issue is the way they write the XML output. The write the output as:
    Code:
    <channel id="1">
    <programme channel="1" />
    <channel id="2">
    <programme channel="2" />
    <channel id="3">
    <programme channel="3" />
    Which is not the correct order. It should be all channel info first then all the program info:
    Code:
    <channel id="1">
    <channel id="2">
    <channel id="3">
    <programme channel="1" />
    <programme channel="2" />
    <programme channel="3" />
    Also, you would have to list the display-name that you want to use first. The first display-name that is listed only has the channel number.

    sub would be able to verify if I am right about the required order.

  4. #4
    Join Date
    Jul 2007
    Location
    USA
    Posts
    166
    Other programs, such as sageTV, do not assume that the channel ID is the channel number. They get the channel number from the fisrt numeric-only <display-name>. In fact, if you look at the official XMLTV description, the station ID they use in their example is the descriptive text ID, not the channel number.
    http://xmltv.cvs.sourceforge.net/xml...td?view=markup

    The XMLTV document specifies that the first display name should perhaps be the channel number, and after that, order them in terms of obscurity.

    Perhaps GBPVR could accept multiple display-names to accomodate the additional information they provide, such as affiliates?

    It depends on how GBPVR is coded as to whether it can deal with the channel IDs not all at the beginning. Most XML COM interfaces wouldn't care the order, someone who has seen the GBPVR source could comment on this...

  5. #5
    Join Date
    Nov 2006
    Location
    Louisville, KY, USA
    Posts
    7,719
    Quote Originally Posted by drlava View Post
    Other programs, such as sageTV, do not assume that the channel ID is the channel number. They get the channel number from the fisrt numeric-only <display-name>. In fact, if you look at the official XMLTV description, the station ID they use in their example is the descriptive text ID, not the channel number.
    Neither does GB-PVR. It will take the channel number from either of the first two display-name tags that you are outputting. The issue is that it ONLY reads the FIRST display-name tag which is only a channel number in your output. I was just lazy in my above examples.

    The main issue is that GB-PVR expects that the channels will all be listed before the programs as per the XMLTV.DTD (shown at http://xmltv.cvs.sourceforge.net/xml...td?view=markup)
    Quote Originally Posted by drlava View Post
    It depends on how GBPVR is coded as to whether it can deal with the channel IDs not all at the beginning. Most XML COM interfaces wouldn't care the order, someone who has seen the GBPVR source could comment on this...
    Only sub would be able to comment on the source. It would not be hard for me to write a program that would rearrange the order of the output file so that it is compatible GB-PVR. If I get some time this weekend I may do that. It would probably be the simplest solution.
    Last edited by whurlston; 2007-08-04 at 05:51 AM.

  6. #6
    Join Date
    Jan 2006
    Posts
    579
    Wow, great find.. this gem looks very promising!
    nPVR 3.1.1 on Win7 x64 Pro|| 46" LG 47LW5300 LCD || Hauppauge HD-PVR2|| Hauppauge HVR-1250
    creator of webStreamer.. now its built-in

  7. #7
    Join Date
    Nov 2006
    Location
    Louisville, KY, USA
    Posts
    7,719
    Ok, I rearranged the output and it turns out that is not the issue after all. The issue is that there is not stop time in the programme tag and the timezone offset is only 3 digits.

    Code:
    <programme channel="28456298" start="20070804000000 -300">
    It should be:

    Code:
    <programme channel="28456298" start="20070804000000 -0300" stop="20070804010000 -0300">
    I can easily fix this but it will be a few days before I can get to it.

  8. #8
    Join Date
    Jul 2007
    Location
    USA
    Posts
    166
    So from the sounds of it only two changes are necessary:?

    I can change the time zone offset to be zero padded to four digits on the source, no problem. The stop is an optional attribute according to the XMLTV documentation, but that could be added in the original source, if GBPVR relies on it. The WiRNS reader uses top or duration to calculate the end time of the programme.

  9. #9
    Join Date
    Nov 2003
    Location
    NextPVR HQ, Wellington, New Zealand
    Posts
    76,563
    It'll cope without the stop time, but of course means the last listing in each channel be dropped since the stop time for these cant be derived.

    <channel id="1">
    <programme channel="1" />
    <channel id="2">
    <programme channel="2" />
    <channel id="3">
    <programme channel="3" />
    It should matter what order the channel or programme tags are supplied, as long as the referenced channel is supplied before the programme, so this example should be ok, though I've never seen an xmltv file in that format so never tested it.

  10. #10
    Join Date
    Nov 2006
    Location
    Louisville, KY, USA
    Posts
    7,719
    Thanks sub. I only changed one or two entries so that may be why it ddnt work for me without the stop time.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •