Changeset 62410 in webkit for trunk/JavaScriptCore/wtf/text/StringImpl.h

Timestamp:

Jul 2, 2010, 3:31:40 PM (15 years ago)

Author:

[email protected]

Message:

2010-07-02 Oliver Hunt <​[email protected]>

Reviewed by Geoffrey Garen.

Move BOM handling out of the lexer and parser
​https://bugs.webkit.org/show_bug.cgi?id=41539

Doing the BOM stripping in the lexer meant that we could
end up having to strip the BOMs from a source multiple times.
To deal with this we now require all strings provided by
a SourceProvider to already have had the BOMs stripped.
This also simplifies some of the lexer logic.

• parser/Lexer.cpp: (JSC::Lexer::setCode): (JSC::Lexer::sourceCode):
• parser/SourceProvider.h: (JSC::SourceProvider::SourceProvider): (JSC::UStringSourceProvider::create): (JSC::UStringSourceProvider::getRange): (JSC::UStringSourceProvider::UStringSourceProvider):
• wtf/text/StringImpl.h: (WebCore::StringImpl::copyStringWithoutBOMs):

2010-07-02 Oliver Hunt <​[email protected]>

Reviewed by Geoffrey Garen.

Move BOM handling out of the lexer and parser
​https://bugs.webkit.org/show_bug.cgi?id=41539

Update WebCore to ensure that SourceProviders don't
produce strings with BOMs in them.

• bindings/js/ScriptSourceProvider.h: (WebCore::ScriptSourceProvider::ScriptSourceProvider):
• bindings/js/StringSourceProvider.h: (WebCore::StringSourceProvider::StringSourceProvider):
• loader/CachedScript.cpp: (WebCore::CachedScript::CachedScript): (WebCore::CachedScript::script):
• loader/CachedScript.h: (WebCore::CachedScript::): CachedScript now stores decoded data with the BOMs stripped, and caches the presence of BOMs across memory purges.

File:

• trunk/JavaScriptCore/wtf/text/StringImpl.h (modified) (1 diff)

trunk/JavaScriptCore/wtf/text/StringImpl.h

@@ -258 +258 @@
     }
 
+    PassRefPtr<StringImpl> copyStringWithoutBOMs(bool definitelyHasBOMs, bool& hasBOMs)
+    {
+        static const UChar byteOrderMark = 0xFEFF;
+        size_t i = 0;
+        if (!definitelyHasBOMs) {
+            hasBOMs = false;
+            // ECMA-262 calls for stripping all Cf characters, but we only strip BOM characters.
+            // See <https://bugs.webkit.org/show_bug.cgi?id=4931> for details.
+            for (; i < m_length; i++) {
+                if (UNLIKELY(m_data[i] == byteOrderMark)) {
+                    hasBOMs = true;
+                    break;
+                }
+            }
+            if (!hasBOMs)
+                return this;
+        }
+        Vector<UChar> result;
+        result.reserveInitialCapacity(m_length);
+        for (; i < m_length; i++)
+            result.append(m_data[i]);
+        for (; i < m_length; i++) {
+            UChar c = m_data[i];
+            if (c != byteOrderMark)
+                result.append(c);
+        }
+        return StringImpl::adopt(result);
+    }
+
     // Returns a StringImpl suitable for use on another thread.
     PassRefPtr<StringImpl> crossThreadString();