Windows 9x Web Helper (9xweb) - Discontinued

[Wunderbar98]

== Windows 9x Web Helper ==

= Overview =

Windows 9x Web Helper is a Bash script, file name 9xweb (25 Kb), designed to improve internet accessibility. Scratch coded using Microsoft Notepad, it is free to use, modify and share, using Unlicense (https://unlicense.org). Tested in vanilla Windows 98. Provided an Enhanced Cygwin-Lite install is functional, it should run on enhanced Windows 98 and is confirmed to run in Windows 95.

= Links =

Overview: Project screenshots, download link and more information. MSFN forum member login required to download.
https://msfn.org/board/topic/177106-running-vanilla-windows-98-in-2020/page/17/?tab=comments#comment-1176154

Pre-requisite: The only pre-requisite is a functional Enhanced Cygwin-Lite install, here's more information and download links.
https://msfn.org/board/topic/177106-running-vanilla-windows-98-in-2020/page/17/?tab=comments#comment-1175895

FAQ:
https://msfn.org/board/topic/181417-windows-9x-web-helper-9xweb/page/3/?tab=comments#comment-1186792

Edited December 14, 2020 by Wunderbar98

[Wunderbar98]

Changelog and milestones:

20200112 Beta - 1st Beta

20200203 Beta - 2nd Beta
- View YT video description
- New sleepSecs variable for file system race condition

20200405 Beta - 3rd Beta
- Improved main screen readability
- Modified video Wget command
- Improved video_fetch and getURL logic
- Remove non-ascii text from video description
- New delete video option after video and information view
- More strategic sleepSecs for file system race condition
- Decreased sleepSecs variable from 3 to 2 seconds
- Additional Video Fetch help
- Video title cleanup

20200411 - Confirmed to work in Windows 95.

20200522 Beta - 4th Beta
- Removed duplicate video_fetch clipboard retrieval
- Modified showClip clipboard cat command (query reduce stackdumps)
- Disabled Video Fetch Wget user agent string (uncomment as desired)
- Modified getHTML and getMedia Wget commands
- Reset several variables after Video Fetch
- Adjusted $sleepSecs instances (file system race condition)
- New $preview variable (off, 5, +5 toggle)
- New $description variable (yes, no, ask)
- Updated Video Fetch [h]elp

20200712 Beta - 5th Beta
- Broke proxy https://proxybrowser.xyz, new http://dontfilter.us
- Quote wrapped all /dev/clipboard instances
- Added '&disable_polymer=1' just in case note
- New $AskKeepvid variable for 'ERROR: 403 Forbidden' YouTube
- Simplified Wget command
- Silenced Wget kill after YouTube preview for clean output
- YouTube title cleanup: excess underscores, period tail
- New YouTube search, specify search or Trending, uses Googlebot UA
- YouTube search results include: video durations, single query cache
- Updated Video Fetch [h]elp

20200805 Beta - 6th Beta
- YouTube 'polymer' compatible: custom search, trending search, video fetch
- Initiate new search directly from search results screen
- More efficient getFormat HTML file chunking
- Modified $sleepSec instances
- Minor UI tweaks

20201124 Beta - 7th Beta
- USER CONFIG $browser2: old seamonkey.exe, new k-meleonw9x.exe
- USER CONFIG $sEngine1: old https://google.com, new http://google.com
- USER CONFIG $sEngine2: old https://search.yahoo.com, new https://lite.duckduckgo.com/lite
- Main screen 'archive.org cache' renamed 'Wayback Machine'
- Function loadUrl improved $url grep
- Function loadUrl now accepts FTP URLs
- Modified newURL function prompt 'enter new URL >OR CONTENT< to clipboard' (ie. bookmark tag)
- Clipboard content now displayed regardless of '--Invalid URL--'
- Clipboard output asthetically trimmed to one line, does not alter actual system clipboard
- Clipboard content >1 line (page break) denoted 'clipboard_content ...'
- Clipboard content with >68 characters denoted 'clipboard_content ..'
- Tag user selected titles from search_YT and cache_YT functions, denoted =>
- Fixed YouTube title extractions containing embedded double quotes
- More efficient $ytURL extraction in get_search_URL function
- More robust and efficient $vidTitle processing in getFormat function
- More efficient $vidText processing in extract_info function
- More robust title grep in search_YT function
- Improved titles cleanup in search_YT function
- Modified Wget commands
- Modified $sleepSecs and sleep instances
- Hopefully fixed pathways and prompts
- Revised Main screen and Video Fetch [h]elp
- Miscellaneous fixes and UI tweaks

Edited November 25, 2020 by Wunderbar98

[Wunderbar98]

Not sure how many more updates but the changelog will be mirrored on post #2 above, and at the first link on post #1. An update is pending, mostly minor changes to fix bugs and improve stability due to file system race condition and Wget occasionally pooping it's pants when fetching videos. Not sure the Wget issue will ever be fully resolved.

[Wunderbar98]

Couple helpful media player information and download links:

SMPlayer/MPlayer and VLC linked below.
https://msfn.org/board/topic/177106-running-vanilla-windows-98-in-2020/page/17/?tab=comments#comment-1176224

TCPMP media player linked below.
https://msfn.org/board/topic/177106-running-vanilla-windows-98-in-2020/page/18/?tab=comments#comment-1176635

[Wunderbar98]

Took another stab at YouTube video streaming, not just downloads. Unfortunately the mkfifo and mknod executables from Enhanced Cygwin-Lite are not functional in Windows 98. Without a FIFO buffer to simultaneously read/write, unable to stream. Tried manual pipes and redirects too, still failed.

Added a partial download preview feature. It will be configurable in the USER CONFIG section of the script and can be toggled and time adjusted during script runtime. Toggling 'preview' off [-] will not launch a preview and just Wgets the entire video. Toggling preview up [+], in 5 second increments, will Wget a partial video for $preview seconds before killing the Wget process. The partial video is then auto-launched by the configured video player for preview. In a video player like TCPMP, the full video duration is displayed in the progress bar but playback progress will obviously stop after it reaches the end of the partial video download.

Wget $preview time allotted, internet connecton speed and video quality preference (640 x 360 vs 1280 x 720) will determine the length/duration of the video preview. On a test system with 640 x 360 video and a basic ethernet connection, setting 'preview' to '5', allows Wget to download for 5 seconds, which provides an approximate 50 second video preview. Typically more than adequate to determine whether the video is desirable for download. After previewing, close the video player, or preferably set video player to auto-close at completion of playback, and either press [Enter] to resume the download or [d]elete the preview and return to the Video Fetch screen. Resuming the download restarts Wget where the preview fetch left off, so no bandwidth gets wasted.

This feature is best suited for longer downloads you're unsure about, check it out before wasting too much time and bandwidth. The code is rough and it will take a few more sessions of testing. The pending update mentioned earlier will incorporate this feature.

Note this preview feature is independent of whether [a]utoplay is toggled on or off, which autoplays the entire video at the end of a full download. These options can easily be toggled during runtime based on user preference. This seems to give lots of choice, preview or not, autoplay or not, depending on whether the goal is to preview, quick hoard a bunch, play and delete, etc.

[siria]

Talking of bandwidth... Not sure if this helps anything, and perhaps mentioned before, just throwing in what very little I know.
There exists also a small 320x180px (funnily called 144p) version as 3gp.
About half file size as medium 360p and of course quite fuzzy. Can play it with old MPC.
Available only on (all black) mobile "m.youtube.com", with e.g. the ipad-UA someone posted here recently
Mozilla/5.0 (iPad; CPU OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25
or IE9-mobile UA: Mozilla/5.0 (compatible; MSIE 9.0; Windows Phone OS 7.5; Trident/5.0; IEMobile/9.0)
or FF38-mobile: Mozilla/5.0 (Android; Mobile; rv:38.0) Gecko/38.0 Firefox/38.0
UA iPad is better since it contains 3 formats incl. 720p, while others gets only 360p+144p

Hmm, could swear IE9-mobile was getting streaming RTSP-links last time I tried...
Or was it 240.mp4? Where has that one vanished now...?!
Anyway, trying to find rtsp again, by UAs, OperaMini still gets it, but ONLY in 144p/3gp:
Opera/9.80 (J2ME/MIDP; Opera Mini/9.80 (J2ME/22.478; U; en) Presto/2.5.25 Version/10.54

HUH.... and although equally called 144p/3gp, this rtsp-version is dramatically smaller as the (fake) 144p/3gp above?! Probably not watchable at all, except for preview checks... Weird, and the screen ratio is totally different too, although from the same video. Don't get it again, oh well - as usual with video stuff.

144p in FF38 or IE9-mobile versus OperaMini:

(http) small 144p = 320 x 180 px
2.32 MB 226 kbps
VIDEO 3GPP (mp4v.20.3) (itag:36)
AUDIO_LOW (mp4a.40.2)

(rtsp) small 144p = 176 x 144 px
0.76 MB 74 kbps
VIDEO 3GPP (mp4v.20.3) (itag:17)
AUDIO_LOW (mp4a.40.2)

(or is this too offtopic here? Can delete it again, no prob)

Edited May 20, 2020 by siria

[Wunderbar98]

Thank-you siria for the information. Originally i filtered different itags and settled on itag 18 (640 x 360 MP4) and itag 22 (1280 x 720 MP4) for simplicity, itag 18 readily available and Windows 98 media players handle MP4.

Personally i find the lower resolution video (320 x 180 and 176 x 144) image quality almost non-functional, unless just using the 'video' to play background music. The lower resolutions probably look okay on a phone or pad, less so on a desktop monitor. IIRC itags 17 and 36 are now rarely available, maybe just because i've been using the wrong user agent string.

Played around a bit, added user agent string below, not sure i got this correct:
Mozilla/5.0 (iPad; CPU OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25

Found a video that provided itag 36, added appropriate fetch code and it downloaded the video. For whatever reason, however, the download was slow, 80 Kb/sec vs the usual 700+ Kb/sec. Interestingly, renaming the 3gp file to mp4 doesn't seem to matter, and all my vanilla Windows 98 media players (VLC, MPlayer, TCPMP) played the video fine, albeit it's poor quality.

Personal preference for itag 18 was outlined above but if there is a demand for lower quality videos i can modify the code. There aren't probably too many using the script but if you are feel free to add a vote.

[Wunderbar98]

Adding a 'View YouTube video description' option. Previously it always displayed but it does require additional processing, not everyone may want it and some may already be able to get this from their browser (not in stock vanilla Windows 98). The default is 'ask' but can be changed in the USER CONFIG section of the script and toggled during script runtime. Options are 'yes', 'no' or 'ask'.

[Wunderbar98]

Still testing before release. Some improvement in Wget reliability and file system race condition. Not sure why Wget sometimes fails, query related to certain YouTube server connections this old Wget can no longer handle. If Wget gives a pop-up error, close the pop-up and try pressing [Enter] on the script. Then retry the video again from the Video Fetch screen. Otherwise, try to gracefully exit the script with [q]uit and 'exit' the COMMAND.COM window. Restart Cygwin-Lite and try again. Last resort just close the COMMAND.COM window with the window X and restart Cygwin-Lite.

Note Video Fetch only requires the YouTube URL, the browser does not need to open to the actual video page on a new tab or anything. So just from a YouTube results search, using RetroZilla's YouTube search engine if desired, just right-click the video you want and select 'Copy Link Location', whatever your browser calls it, to get the URL to the clipboard.

Websites with embedded YouTube videos will fail when using old browsers with JavaScript disabled. Depending on the browser, it will likely display a black YouTube video box with the message 'An error occurred. Try watching this video on www.youtube.com, or enable JavaScript if it is disabled in your browser'. No problem, just right-click the embedded link portion labelled 'Try watching this video on www.youtube.com' and copy the link to clipboard. There is no need to open a new tab or visit the actual YouTube page.

[Wunderbar98]

Experienced occasional (rare) TCPMP crash with MP4 playback. This does not appear video related, as the same file plays back without error on other media players (VLC, SMPlayer/MPlayer). As this has only been tested and experienced on one system, query whether it is this system's particular configuration. If TCPMP is problematic, feel free to configure a different video player in the 9xweb script's USER CONFIG section. Not sure if forum member @roytam1 modified TCPMP's code or just shared the binary, whether it is worth submitting the crash text for review.

Quick tip: The main Video Fetch screen automatically seeks the system's clipboard contents upon finishing a previous video. So if you play a video, right-click and select a new video URL from the web browser, then select 'Press [Enter] to continue' in the 9xweb script, the main Video Fetch screen displays the updated URL, no need to manually [r]efresh the clipboard.

Quick tip: Toggling settings in the main Video Fetch screen also auto-refreshes the clipboard content, such as changing preview or autoplay settings.

Hope to upload a new 9xweb script today or tomorrow.

[Wunderbar98]

A new 9xweb download is available from the first link on the first post of this thread (forum login required). A new Video Fetch screenshot was posted, neglected to upload a new main screen screenshot. The script version is 20200522 Beta. The Changelog post #2 above was also updated.

[Wunderbar98]

Came across a site yesterday that can download YouTube videos via webpage. It does not appear to get them all, had a VEVO video still fail, but seems to get most, many of which would be inaccessible to my 9xweb script.

Pros:
- Simply paste YouTube URL into webpage
- Fast download speed
- Does not require browser JavaScript
- Works in RetroZilla

Cons:
- Does not provide video information
- Does not auto-name video (without JS anyway)

Bookmarked the site in RetroZilla for now. Not sure yet if i will incorporate a link to this site in my 9xweb script for videos that otherwise fail. It would definitely improve YouTube access for more vanilla Windows 9x systems. Just like youtube-dl, YouTube keeps changing things and the site can sometimes fail, as outlined in the home page's changelog.

keepvid[dot]works

Took another look at youtube-dl's code for YouTube, not sure yet how they get VEVO type videos to work. Brain tired, may chip away again sometime later, later, later.

[roytam1]

On 5/23/2020 at 12:10 PM, Wunderbar98 said:

Not sure if forum member @roytam1 modified TCPMP's code or just shared the binary

actually the later, but I had its source code (isn't figure how to compile it at the moment) uploaded to github:

https://github.com/roytam1/TCPMP

[siria]

Wunderbar98 said:
> Came across a site yesterday that can download YouTube videos via webpage.
> It does not appear to get them all, had a VEVO video still fail, but seems to get most,
> many of which would be inaccessible to my 9xweb script.

Thanks for the tip :) Awhile back tried to get a video as mp3, but no chance. Next time will try keepvid.
Just curious if my own scripts works with those 'inaccessible' vids, if you can post links?
But can only download allowed videos too, not protected ones.
(PS: sorry but I cannot run cygwin or anything, zero chance)

Edited June 20, 2020 by siria

[Wunderbar98]

Thank-you for the feedback and link @roytam1. If you start re-compiling TCPMP i can submit a crash.txt report.

Hi @siria. Not sure why you can't run Cygwin-Lite, it's not hard to install and set up. If your script is running reliably, consider sharing as a new member project.

Three MP4 links to check, all work fine in GNU/Linux using youtube-dl.

9xweb 403 forbidden error, also FAILS with keepvid[dot]works:
https://www.youtube.com/watch?v=dBF78tA443A
https://www.youtube.com/watch?v=iuzyA5gDa4E

9xweb 403 forbidden error, but WORKS with keepvid[dot]works:
https://www.youtube.com/watch?v=LCCiwPEdEpg