Jump to content
MSFN is made available via donations, subscriptions and advertising revenue. The use of ad-blocking software hurts the site. Please disable ad-blocking software or set an exception for MSFN. ×

My Browser Builds (Part 3)


roytam1
 Share

Recommended Posts


@mixit, @roytam1

Just out of curiosity, roughly how much disk space is needed to compile a browser currently (Serpent52/NM27/NM28/Centaury) - source + dependencies + intermediate files created when building ?

Don't count the compilers/IDE/etc.

Link to comment
Share on other sites

9 hours ago, mixit said:

I've no idea how Mozilla managed to keep this bug around ever since Firefox 15.0

Don't forget that Google knew about such problems and abused that knowledge, though Google coders working with Mozilla team should rather help to make fixes. They even might deliberately keep bugs unfixed.

Mozilla claims Google has made YouTube perform worse on Edge and Firefox

Mozilla engineer says Google slowed YouTube down on non-Chrome browsers

Former Mozilla exec: Google has sabotaged Firefox for years

 

  • Like 1
Link to comment
Share on other sites

Nothing really new - they have all done it over the years.

Android was once caught rigging their phones to falsely score high on very popular "benchmarks" but would perform poorly in "real-world" scenarios.

And Android got caught doing that just a few short years after they themselves caught Samsung cheating benchmarks on Note 3 and Galaxy S4.

AMD's entire Athlon line of CPUs were all premised on "We know the clock is only 1.8 GHz, but it performs equally to a 2.8 GHz" (which they did, btw, I was an AMD over-clocker for years) - though I guess that's more "marketing" then "cheating".

Bing used to always load faster on IE then it did in Firefox.

Google probaby used to do the same thing, but no reason to anymore, isn't Google the default search engine for Firefox?

It's probably more of a smartphone thing nowadays, cheat your "score" for the sake of sales.

Though I have always felt that if the "turbo engine" was marketed as "fuel-efficient" instead of "performance", then all cars would have turbos in the vein of "going green".

"Why did you buy such a fast car?"  "I didn't buy it because it's fast, I bought it because it's the most fuel-efficient in its class."

  • Like 1
Link to comment
Share on other sites

Hi guys, if I want to apply mixit's fix on Firefox ESR 52.9.0 for example, where do I need to put this file cubeb_winmm.c? A little help?

Link to comment
Share on other sites

On 7/26/2021 at 5:00 PM, RainyShadow said:

@mixit, @roytam1

Just out of curiosity, roughly how much disk space is needed to compile a browser currently (Serpent52/NM27/NM28/Centaury) - source + dependencies + intermediate files created when building ?

Don't count the compilers/IDE/etc.

Let's see here... Looks like my latest Centaury build uses roughly 6.2 GB altogether, 1.4 GB for the source and 4.8 GB for the "object directory" with build results. This is a release build, debug would be somewhat larger - so let's say 7-8 GB in total.

MozillaBuild is roughly 0.8 GB. The full DirectX SDK is 1.2 GB installed (although you really only need a few MB from it). For Windows 10 EWDK based builds (like @feodor2 does them) the EWDK .iso (that you can mount as-is) is 6-9 GB depending on version (getting bigger), but you don't have to install VS and the Windows SDK, so you can still save quite a bit that way.

It was pretty funny to see Moonchild arrogantly lecture feodor2 that he didn't "understand how software in general works" because feodor2 didn't want to install all the extra bloat that comes with VS, whereas Mozilla have been using stripped down tools packages since well before Firefox 52.0 that UXP is based on :P (see the history of their toolchain packaging script). (OK, so they do the install once, but then package the components they actually need and use that package for subsequent builds.) If you did that (installed, repackaged, uninstalled) with the current official MCP requirements for Pale Moon/Basilisk, you'd end up with only 1.3 GB for all the required VS and SDK stuff, huge space savings!
 

On 7/26/2021 at 6:49 PM, we3fan said:

Thanks for this mixit.

How to implement this fix?

You're very welcome :) Since I don't plan on distributing my own builds publicly, I guess you'll have to wait until @roytam1 and/or @feodor2 merge this into their builds, so quite possibly only until this week-end for Serpent (although they'll likely want to spend a bit more time testing this on their own).

 

On 7/27/2021 at 8:37 AM, we3fan said:

Hi guys, if I want to apply mixit's fix on Firefox ESR 52.9.0 for example, where do I need to put this file cubeb_winmm.c? A little help?

You'd need to rebuild the whole application, replacing media/libcubeb/src/cubeb_winmm.c in the source code. Unfortunately this is not something that can be applied "on-the-fly".

Edited by mixit
  • Like 3
  • Upvote 1
Link to comment
Share on other sites

47 minutes ago, roytam1 said:

LGTM :thumbup

will commit it in short period.

EDIT: committed in goanna3 and 4 repo, other repos will be followed later.

Cool, thanks! :D And you even gave (ample!) proper credit and everything, quite unlike the "thieving hacker" image certain folks at MCP like to project of you, lol! :whistle:

Edited by mixit
  • Like 4
Link to comment
Share on other sites

kind-of off-topic:

lately I'm a bit busy on FLOSS project(s). For example, I co-op with mksh's author to port mksh to SCO UNIX 3.2:
https://github.com/MirBSD/mksh/compare/ec07d019c758de5a457e6e2e896cc6314fa58d24...345f09eb1d9bc3879ebce9728ed71a762a36c4ea
and hoping to port(kind-of) to Xenix.

or raising suggestions on https://github.com/jhhoward/MicroWeb/issues

Link to comment
Share on other sites

On 7/25/2021 at 8:48 PM, Usher said:

For 720p and higher Youtube uses MPEG DASH standard - it downloads separately audio and video streams divided in segments.

Actually YouTube offers several format options for 720p. Format 22 is a single file.

E.g. for this video (found formats with yt-dlp, using the following command):

yt-dlp -F https://www.youtube.com/watch?v=I79CQUTO5II


 

22  mp4  1280x720   30  |             852k https | avc1.64001F    852k mp4a.40.2   0k 44100Hz 720p
136 mp4  1280x720   30  |  89.77MiB   723k https | avc1.4d401f    723k                        720p (throttled), mp4_dash
247 webm 1280x720   30  |  88.94MiB   716k https | vp9            716k                        720p (throttled), webm_dash
398 mp4  1280x720   60  |  108.10MiB  871k https | av01.0.08M.08  871k                        720p60, mp4_dash


480p however has only Dash:

397 mp4  854x480    30  |  42.61MiB   343k https | av01.0.04M.08  343k                        480p, mp4_dash
135 mp4  854x480    30  |  29.41MiB   236k https | avc1.4d401f    236k                        480p, mp4_dash
244 webm 854x480    30  |  32.46MiB   261k https | vp9            261k                        480p, webm_dash

But note "throttled" in the above output. Maybe that can be the culprit?

cc @we3fan

Edited by nicolaasjan
  • Like 2
Link to comment
Share on other sites

Thanks for the detailed answer, @mixit

I already have VS 2019 and DirectX SDK installed, part of the reason i'm left with less than 40GB space on the C: drive, lol.

Hopefully using another drive won't mess things if/when i try building something bigger (like a browser) sometime in the far future.

Still no clue what i'm doing in that damn IDE, especially when a project won't build as is, but needs some fixing first.

A simple "make config / make install" or the Arduino IDE come a lot more easier.

 

P.S. there is HttpDisk, which allows mounting iso and disk images over the internet, no need to download several GBs to extract only a few files.

Edited by RainyShadow
  • Like 1
Link to comment
Share on other sites

On 7/25/2021 at 10:14 AM, we3fan said:

Hi @modnar, I am not sure if we talk about the same issue.

When I watch youtube video with Firefox or Firefox-derived browser on 720p, often the audio will continue but the image will freeze for 7 seconds. This doesn't happen on 480p.

I have 2GB RAM, maybe upgrade to 4GB RAM would fix this, not sure.

Just to answer - I experience a ~16s freeze but I don't want to lower quality because of that, so 4GB of RAM doesn't help really. As @mixit said it's an audio bug in the library...

I have 4GB of RAM and a GTX 980 video card with SBAudigy2 and this should be enough for all my audio and video needs up to 1080p60fps (though I use 1680x1050 screen) and it works and now this fix - wow - something to look for in the next Serpent build. Mighty fine. :thumbup

 

On 7/26/2021 at 10:33 AM, mixit said:

I guess this is my cue to stop procrastinating and finally post the fix I made for this :) I've had a few people test it for 4 months now on all kinds of sites on both XP x86 and x64, and the jury says this indeed fixes the 2x:xx video stoppage issue and doesn't seem to introduce any new problems. I honestly didn't intend for the testing period to be quite this long, but summer heat can have a detrimental effect on one's brain and thus plans. :D

Since my own browser builds are based on Centaury, but I still consider myself a (lurking) MSFN patriot and this is the home of @roytam1's Serpent, I'm just going to post the fixed media/libcubeb/src/cubeb_winmm.c on Pastebin to avoid playing favorites (and signing up on Github ;)) I'm not sure how often @feodor2 visits here these days, so maybe someone on Github could mention this in https://github.com/Feodor2/Mypal/issues/1 if he doesn't pick this up soon enough.

What has been tested:

  • MP4/VP9/VP8 (and plain audio MP3/etc)
  • streaming/local
  • chunked/single-file
  • sped-up/slowed-down
  • 27+ hrs long playback (see the technical details for why)
  • Youtube/Twitter/Facebook/Instagram/TV station streams/etc/etc (nor did we forget p0rn/pirate sites, which of course are no different from a technical POV... :P).
  • Among other things, a 75-year old lady watched the entirety of Prince Philip's funeral with this fix in effect and had no complaints. :D

What has NOT been tested:

  • Pale Moon-based builds (as opposed to Basilisk), because I don't use NM or Mypal. Since this is a UXP platform level fix, the front-end should have no effect.
  • I obviously don't have access to every sound card out there, but since the MS driver causing these problems should be common in all configurations, I'd expect the fix to work with pretty much every card.
  • new-fangled formats like AV1 (I'd expect them work, though).
  • DRM-ed streams, since no Widevine on XP (but again, this should work with those as well if DRM itself worked).

 @roytam1, @feodor2 I didn't create a preference for turning this fix off, because it turned out the normal pref system is no longer compatible with C code, and once it became evident enough that the fix was pretty solid, I didn't feel like hacking something together to make preffing work for a library-level change like this. You're welcome to pref it, of course, but based on the length of testing with no issues discovered, I dare claim it's safe to include without a pref.

Here's a general overview of why this problem was happening (you may be surprised :ph34r:):

  Reveal hidden contents

Basically, this was always an audio problem, even though it manifested as a video freeze. The idea is that an A/V stream (in this case, "stream" means linear playback starting from any position with possible pausing, but no seeking/jumping in either direction) should always move forward, but due to an MS audio driver bug the audio position counter overflows (effectively resets) and appears to move backward. The coupled video stream sees this, expects it to be a short-term stutter issue and basically goes "OK, I'll wait a bit until the audio starts moving forward again" - and ends up staying like this, because the audio can't get its reported position past the overflow point. Since, ironically, this doesn't actually affect audio playback, the user reasonably but wrongly assumes that it's a video issue...

I have to say I was always curious about this issue, but since it was a minor annoyance and easy enough to work around wirh a couple of key presses, I never felt compelled enough to spend time setting up the Mozilla build environment to really get into it while official Firefox was still XP compatible. But, since I've started doing my own builds now anyway, there was no longer any excuse not to investigate this. I was pleased to discover that my hunch about this being an overflow issue was correct, although I never considered the possibility of audio being the actual culprit. I've no idea how Mozilla managed to keep this bug around ever since Firefox 15.0 when it was first introduced by setting cubeb as the default audio library and deprecating libsydneyaudio (or, rather, it was made more manifest; it had also existed with libsydneyaudio in a much less obvious form, occurring roughly after every 3 hrs).

And, some more technical specifics, incl. about where the 2x:xx times come from (this is also included as a comment in the source code):

  Reveal hidden contents

Microsoft wave audio docs say "samples are the preferred time format in which to represent the current position", but relying on this causes problems on Windows XP, the only OS cubeb_winmm is used on.

While the wdmaud.sys driver internally tracks a 64-bit position and ensures no backward movement, the WinMM API limits the position returned from waveOutGetPosition() to a 32-bit DWORD (this applies equally to XP x64). The higher 32 bits are chopped off, and to an API consumer the position can appear to move backward.

In theory, even a 32-bit TIME_SAMPLES position should provide plenty of playback time for typical use cases before this pseudo wrap-around, e.g:

(2^32 - 1)/48000 = ~24:51:18 for 48.0 kHz stereo;
(2^32 - 1)/44100 = ~27:03:12 for 44.1 kHz stereo.

In reality, wdmaud.sys doesn't provide a TIME_SAMPLES position at all, only a 32-bit TIME_BYTES position, from which wdmaud.drv derives TIME_SAMPLES:

SamplePos = (BytePos * 8) / BitsPerFrame,
where BitsPerFrame = Channels * BitsPerSample,

Per dom\media\AudioSampleFormat.h, desktop builds always use 32-bit FLOAT32 samples, so the maximum for TIME_SAMPLES should be:

(2^29 - 1)/48000 = ~03:06:25;
(2^29 - 1)/44100 = ~03:22:54.

This might still be OK for typical browser usage, but there's also a bug in the formula above: BytePos * 8 (BytePos << 3) is done on a 32-bit BytePos, without first casting it to 64 bits, so the highest 3 bits, if set, would get shifted out, and the maximum possible TIME_SAMPLES drops unacceptably low

(2^26 - 1)/48000 = ~00:23:18;
(2^26 - 1)/44100 = ~00:25:22.

To work around these limitations, we just get the position in TIME_BYTES, recover the 64-bit value, and do our own conversion to samples.

EDIT: Writing this up amply reminded me how much I LOVE :no: making long posts using this board's post editor... BBCode FTW!:thumbup

Hallelujah!!!

Now this is resolved! I'm very thankful for your time in fixing and testing and providing a fix, @mixit, for our favorite browser and of course @roytam1 for implementing it in all XP browsers!

WOOHOO! SuperSweet! Shaderlicious and such. :D:cool::thumbup

  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   1 member


×
×
  • Create New...