Changeset 62449 in webkit for trunk/JavaScriptCore/wtf/text/StringImpl.h

Timestamp:

Jul 3, 2010, 1:30:24 PM (15 years ago)

Author:

[email protected]

Message:

Move BOM handling out of the lexer and parser
​https://bugs.webkit.org/show_bug.cgi?id=41539

Reviewed by Geoffrey Garen.

JavaScriptCore:

Doing the BOM stripping in the lexer meant that we could
end up having to strip the BOMs from a source multiple times.
To deal with this we now require all strings provided by
a SourceProvider to already have had the BOMs stripped.
This also simplifies some of the lexer logic.

• parser/Lexer.cpp:

(JSC::Lexer::setCode):
(JSC::Lexer::sourceCode):

• parser/SourceProvider.h:

(JSC::SourceProvider::SourceProvider):
(JSC::UStringSourceProvider::create):
(JSC::UStringSourceProvider::getRange):
(JSC::UStringSourceProvider::UStringSourceProvider):

• wtf/text/StringImpl.h:

(WebCore::StringImpl::copyStringWithoutBOMs):

WebCore:

Update WebCore to ensure that SourceProviders don't
produce strings with BOMs in them.

• bindings/js/ScriptSourceProvider.h:

(WebCore::ScriptSourceProvider::ScriptSourceProvider):

• bindings/js/StringSourceProvider.h:

(WebCore::StringSourceProvider::StringSourceProvider):

• loader/CachedScript.cpp:

(WebCore::CachedScript::CachedScript):
(WebCore::CachedScript::script):

• loader/CachedScript.h:

(WebCore::CachedScript::):

CachedScript now stores decoded data with the BOMs stripped,
and caches the presence of BOMs across memory purges.

File:

• trunk/JavaScriptCore/wtf/text/StringImpl.h (modified) (1 diff)

trunk/JavaScriptCore/wtf/text/StringImpl.h

@@ -258 +258 @@
     }
 
+    PassRefPtr<StringImpl> copyStringWithoutBOMs(bool definitelyHasBOMs, bool& hasBOMs)
+    {
+        static const UChar byteOrderMark = 0xFEFF;
+        size_t i = 0;
+        if (!definitelyHasBOMs) {
+            hasBOMs = false;
+            // ECMA-262 calls for stripping all Cf characters, but we only strip BOM characters.
+            // See <https://bugs.webkit.org/show_bug.cgi?id=4931> for details.
+            for (; i < m_length; i++) {
+                if (UNLIKELY(m_data[i] == byteOrderMark)) {
+                    hasBOMs = true;
+                    break;
+                }
+            }
+            if (!hasBOMs)
+                return this;
+        }
+        Vector<UChar> result;
+        result.reserveInitialCapacity(m_length);
+        size_t firstBOM = i;
+        i = 0;
+        for (; i < firstBOM; i++)
+            result.append(m_data[i]);
+        for (; i < m_length; i++) {
+            UChar c = m_data[i];
+            if (c != byteOrderMark)
+                result.append(c);
+        }
+        return StringImpl::adopt(result);
+    }
+
     // Returns a StringImpl suitable for use on another thread.
     PassRefPtr<StringImpl> crossThreadString();