Bashpodder trouble

[ChrisULM]

I've been messing around with bashpodder on my server for the last hour or so. I've got it all setup to run automatically using a cron job, and for the most part its working. However, for some reason the latest episode of diggnation (138) downloads as an "index.html" file instead of the video. All the other shows on Revision3 download fine as well as the techshow. Could this be a problem on my end or is it something on revision3's end?

Thanks for any help you can give.

[davijordan]

show us the code!

[dann]

This could be a problem with they way some sites use redirect's for round robin file hosting. In addition to posting your code could you also post a link to the feed you use for diggnation. Yeah, I'm too lazy to go fetch it myself.

[ChrisULM]

I'm using one of the variants that somebody posted here - http://lincgeek.org/bashpodder/user_con ... _hefferan/
Diggnation RSS:
http://revision3.com/diggnation/feed/quicktime-large/

Bashpodder Code:

Code: Select all

#!/bin/bash
# By Linc 10/1/2004
# Find the latest script at http://linc.homeunix.org:8080/scripts/bashpodder
# If you use this and have made improvements or have comments
# drop me an email at linc dot fessenden at gmail dot com
# I'd appreciate it!
#
# This revision by Brian Hefferan 2004/02/06, adding configuration options.
# No warranty. It seems to work for me, I hope it works for you.
# Questions /corrections on the additions by Brian Hefferan can be sent to
# brian at heftone  dot  com

#default values can be set here. Command-line flags override theses.
verbose=
wget_quiet='-q'  #default is -q
wget_continue=
catchup_all=
first_only=
unix2dos=
usetorrents=
sync_disks=
fetchlist='bp.conf'

function usage
{
  echo "
Usage: $0 [OPTIONS]
Options are:
-v, --verbose          display verbose messages. Also enables wget's continue
                      option.
--catchup_all          write all urls to the log file without downloading the
                      actual podcasts. This is useful if you want to subscribe
                      to some podcasts but don't want to download all the back
                      issues. You can edit the podcast.log file afterwards to
                      delete any url you still wish to download next time
                      bashpodder is run.
--first_only           grab only the first new enclosed file found in each feed.
                      The --catchup_all flag won't work with this option. If
                      you want to download the first file and also permanently
                      ignore the other files, run bashpodder with this option,
                      and then run it again with --catchup_all.
-bt --bittorrent       launch bittorrent for any .torrent files downloaded.
                      Bittorrent must be installed for this to work. The
                      the script and bittorrent process will continue running
                      in the foreground indefinitely. You can use ctr-c to
                      kill it when you want to stop participating in the
                      torrent.
--sync_disks           run the "sync" command twice when finished. This helps
                      makes sure all data is written to disk. Recommended if
                      data is being written directly to a portable player or
                      other removable media.
-u, --url_list         ignore bp.conf, instead use url(s) provided on the
                      command line. The urls should point to rss feeds.
                      If used, this needs to be the last option on the
                      command line. This can be used to quickly download just
                      a favorite podcast, or to take a few new podcasts for a
                      trial spin.
-h, --help             display this help message

"
}

if [ -n "$verbose" ]; then wget_quiet='';wget_continue='-c';fi
if test -f urls.temp;then rm urls.temp;fi

# Make script crontab friendly:
cd $(dirname $0)

while [ "$1" != "" ];do
   case $1 in
             -v|--verbose ) verbose=1
                            wget_continue='-c'
                            wget_quiet=''
                         ;;
            -u|--url_list ) shift
                            while [ "$1" != "" ];do
                               echo "$1" >> urls.temp
                               shift
                            done
                            if test ! -f urls.temp
                               then
                                   echo "Error: -u or --url_list option specified, but no urls given on command line. quitting."
                                   exit 1;
                            fi
                            fetchlist='urls.temp'
                         ;;
            --catchup_all ) catchup_all=1
                         ;;
             --first_only ) first_only=1
                         ;;
             --bittorrent ) usetorrents=1
                         ;;
             --sync_disks ) sync_disks=1
                         ;;
                -h|--help ) usage
                            exit
                         ;;
   esac
   shift
done

# datadir is the directory you want podcasts saved to:
datadir='/home/chris/Vidcasts'

# Check for and create datadir if necessary:
if test ! -d $datadir
      then
      mkdir $datadir
fi

if test ! -f bp.conf && test ! -f urls.temp;
then
   echo "Sorry no bp.conf found, and no urls in command line. Run $0 -h for usage."
   exit
fi

# Read the bp.conf file and wget any url not already in the podcast.log file:
while read podcast
      do
      seenfirst=
      if [ -n "$verbose" ]; then echo "fetching rss $podcast...";fi;
      for url in $(wget -q "$podcast" -O - | tr '\r' '\n' | tr \' \" | \
                   sed -n 's/.*url *= *"\([^"]*\)".*/\1/p' )
              do
          if [ -n "$first_only" ] && [ -n "$seenfirst" ]; then break;fi
          echo $url >> temp.log
          if [ -n "$catchup_all" ];
          then
              if [ -n "$verbose" ]; then echo " catching up $url...";fi
          elif   ! grep "$url" podcast.log > /dev/null ;
          then
             if [ -n "$verbose" ]; then echo "  downloading $url...";fi
             wget $wget_continue $wget_quiet -P $datadir "$url"
          fi
          seenfirst=1
      done
done < $fetchlist

if test ! -f temp.log && [ -n "$verbose" ];then echo "nothing to download.";fi

if test -f urls.temp; then rm urls.temp;fi

# Move dynamically created log file to permanent log file:
cat podcast.log >> temp.log
sort temp.log | uniq > podcast.log
rm temp.log

# Use bittorrent to download any files pointed from bittorrent files:
if [ "$usetorrents" ]
then
    if ls $datadir/*.torrent 2> /dev/null
    then
          btlaunchmany.py $datadir
    fi
fi

# Create an m3u playlist:
ls -1rc $datadir | grep -v m3u > $datadir/podcast${datadir}.m3u
if [ -n "$unix2dos" ];then unix2dos $datadir/podcast${datadir}.m3u;fi;

if [ -n "$sync_disks" ]
then
    if [ -n "$verbose" ]; then echo "running sync..";fi;
    sync
    if [ -n "$verbose" ]; then echo "running sync again..";fi;
    sync
fi

if [ -n "$verbose" ]; then echo "done.";fi;

Thanks for the help guys

[Ryochan7]

When I run that script, it grabs the videos twice as well as screenshots and the html pages for each episode. The script grabs all elements from tags with url= in it. Several of the tags, besides enclosure, in the diggnation feed use url= in it. To fix the problem, you need to correct the sed command that parses the feed to get the urls so that it only grabs urls from enclosure tags.

Change sed command on line 123 from

Code: Select all

sed -n 's/.*url *= *"\([^"]*\)".*/\1/p'

to

Code: Select all

sed -n 's/.*enclosure.*url="\([^"]*\)".*/\1/p'

[ChrisULM]

That seems to have worked!
Thank you so much for the help. I really appreciate it.

[lambretta]

Hi guys, enjoy the show and think bashpodder is the shitzen - so light - perfect for my DNS-323 NAS.

Sorry to bring up an old thread but I am also having some problems getting bashpodder to download all the enclosures from an RSS feed.

Here is the feed URL that I am using, its from mininova, I am trying to use it to download a .torrent file into a folder for rtorrent to action.....
http://www.mininova.org/rss.xml?user=FinalGear&num=5

Here is a snip of the script that I am using (its the Brian Heffernan alteration to the original bashpodder script) - I notice Linc has just yesterday removed the old changes and put a link to his forum instead

Code: Select all

# Read the bp.conf file and wget any url not already in the podcast.log file:
while read podcast
      do
      seenfirst=
      if [ -n "$verbose" ]; then echo "fetching rss $podcast...";fi;
      for url in $(wget -q "$podcast" -O - | tr '\r' '\n' | tr \' \" | \
                   sed -n 's/.*url *= *"\([^"]*\)".*/\1/p' )
              do
          if [ -n "$first_only" ] && [ -n "$seenfirst" ]; then break;fi
          echo $url >> temp.log
          if [ -n "$catchup_all" ];

I am guessing that the RSS feed isn't in accordance with what the parse_enclosure.xls stylesheet is expecting. Is this this correct?

If I change the feed to only show one result like this http://www.mininova.org/rss.xml?user=FinalGear&num=1 it seems to work OK.

Any way to fix this other than limiting feed to one result?

[Linc]

Hi guys, enjoy the show and think bashpodder is the shitzen - so light - perfect for my DNS-323 NAS.

Sorry to bring up an old thread but I am also having some problems getting bashpodder to download all the enclosures from an RSS feed.

Here is the feed URL that I am using, its from mininova, I am trying to use it to download a .torrent file into a folder for rtorrent to action.....
http://www.mininova.org/rss.xml?user=FinalGear&num=5

Here is a snip of the script that I am using (its the Brian Heffernan alteration to the original bashpodder script) - I notice Linc has just yesterday removed the old changes and put a link to his forum instead

Code: Select all

# Read the bp.conf file and wget any url not already in the podcast.log file:
while read podcast
      do
      seenfirst=
      if [ -n "$verbose" ]; then echo "fetching rss $podcast...";fi;
      for url in $(wget -q "$podcast" -O - | tr '\r' '\n' | tr \' \" | \
                   sed -n 's/.*url *= *"\([^"]*\)".*/\1/p' )
              do
          if [ -n "$first_only" ] && [ -n "$seenfirst" ]; then break;fi
          echo $url >> temp.log
          if [ -n "$catchup_all" ];

I am guessing that the RSS feed isn't in accordance with what the parse_enclosure.xls stylesheet is expecting. Is this this correct?

If I change the feed to only show one result like this http://www.mininova.org/rss.xml?user=FinalGear&num=1 it seems to work OK.

Any way to fix this other than limiting feed to one result?

Yup. When I run that through the current BashPodder I get a pile of files named:

• 1927026
  1944448
  1944467
  1950946
  1950950
  1950953
  1970985
  1972566
  1993980
  1999679
  2016297
  2018221
  2034411
  2053185
  2073814
  2093301
  2128752
  2151104
  2173202
  2195972

Although the torrent names don't come through very well, the files themselves still appear to be downloaded. Renaming them should be trivial.

Always use the latest version of BashPodder

[lambretta]

Thanks Linc, will use the current one.

Alas I wish it were trivial for me to modify the script to automatically change the name of the downloaded file to name_of_file.torrent instead of just the number.

I have looked at the old script I'm using and compared it to the the current script on your site but it isn't obvious to me what changes the names of the enclosure to name_of_file.torrent in the old script.

Any pointers on what needs changing?

[Linc]

Thanks Linc, will use the current one.

Alas I wish it were trivial for me to modify the script to automatically change the name of the downloaded file to name_of_file.torrent instead of just the number.

I have looked at the old script I'm using and compared it to the the current script on your site but it isn't obvious to me what changes the names of the enclosure to name_of_file.torrent in the old script.

Any pointers on what needs changing?

Yes, they need to name their files what they want them to be called.