Commit Graph

87 Commits

Author SHA1 Message Date
Timothy Gu
99b27ce99a url: prioritize toString when stringifying
The ES addition operator calls the ToPrimitive() abstract operation
without hint String, leading a subsequent OrdinaryToPrimitive() to call
valueOf() first on an object rather than the desired toString().

Instead, use template literals which directly call ToString() abstract
operation, per Web IDL spec.

PR-URL: https://github.com/nodejs/node/pull/11737
Fixes: b610a4db1c "url: enforce valid UTF-8 in WHATWG parser"
Refs: b610a4db1c (commitcomment-21200056)
Refs: https://tc39.github.io/ecma262/#sec-addition-operator-plus-runtime-semantics-evaluation
Refs: https://tc39.github.io/ecma262/#sec-template-literals-runtime-semantics-evaluation
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2017-03-10 20:02:24 -08:00
Brian White
7afc70de40
url: remove invalid file protocol check
'file' should have been 'file:' but since the WHATWG URL spec
suggests using an opaque origin (which is what was already being
used for file URLs), we'll just keep using that, making this merely
a cleanup.

PR-URL: https://github.com/nodejs/node/pull/11691
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Timothy Gu <timothygu99@gmail.com>
Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2017-03-09 08:56:51 -05:00
Timothy Gu
b610a4db1c url: enforce valid UTF-8 in WHATWG parser
This commit implements the Web IDL USVString conversion, which mandates
all unpaired Unicode surrogates be turned into U+FFFD REPLACEMENT
CHARACTER. It also disallows Symbols to be used as USVString per spec.

Certain functions call into C++ methods in the binding that use the
Utf8Value class to access string arguments. Utf8Value already does the
normalization using V8's String::Write, so in those cases, instead of
doing the full USVString normalization, only a symbol check is done
(`'' + val`, which uses ES's ToString, versus `String()` which has
special provisions for symbols).

PR-URL: https://github.com/nodejs/node/pull/11436
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
2017-02-28 18:39:20 -08:00
Timothy Gu
fa41dd117d url: fix handling of ? in URLSearchParams creation
PR-URL: https://github.com/nodejs/node/pull/11372
Fixes: https://github.com/nodejs/node/issues/11093
Ref: https://github.com/whatwg/url/issues/248
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2017-02-16 21:12:23 -08:00
Michaël Zasso
66e263035a url: implement URL.prototype.toJSON
PR-URL: https://github.com/nodejs/node/pull/11236
Ref: https://github.com/whatwg/url/pull/229
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Timothy Gu <timothygu99@gmail.com>
2017-02-16 12:34:09 -08:00
Timothy Gu
c6b12d0984 url: fix surrogate handling in encodeAuth()
Also factor out common parts in querystring and url.

PR-URL: https://github.com/nodejs/node/pull/11161
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
2017-02-14 12:13:11 -08:00
Timothy Gu
781eb90960 benchmark: add url/url-searchparams-sort.js
PR-URL: https://github.com/nodejs/node/pull/11098
Fixes: https://github.com/nodejs/node/issues/10760
Ref: https://github.com/whatwg/url/issues/26
Ref: https://github.com/whatwg/url/pull/199
Ref: https://github.com/w3c/web-platform-tests/pull/4531
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2017-02-13 20:49:13 -08:00
Timothy Gu
02d1e32fe3 url: add urlSearchParams.sort()
PR-URL: https://github.com/nodejs/node/pull/11098
Fixes: https://github.com/nodejs/node/issues/10760
Ref: https://github.com/whatwg/url/issues/26
Ref: https://github.com/whatwg/url/pull/199
Ref: https://github.com/w3c/web-platform-tests/pull/4531
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2017-02-13 20:48:58 -08:00
James M Snell
9549329158 fs: allow WHATWG URL and file: URLs as paths
Updates the fs module APIs to allow 'file://' URL objects
to be passed as the path.

For example:

```js
const URL = require('url').URL;
const myURL = new URL('file:///C:/path/to/file');
fs.readFile(myURL, (err, data) => {});
```

On Windows, file: URLs with a hostname convert to UNC paths,
while file: URLs with drive letters convert to local absolute
paths:

```
file://hostname/a/b/c => \\hostname\a\b\c
file:///c:/a/b/c => c:\a\b\c
```

On all other platforms, file: URLs with a hostname are unsupported
and will result in a throw:

```
file://hostname/a/b/c => throw!
file:///a/b/c => /a/b/c
```

The documentation for the fs API is intentionally not updated in
this commit because the URL API is still considered experimental
and is not officially documented *at this time*

Note that file: URLs are *required* by spec to always be absolute
paths from the file system root.

This is a semver-major commit because it changes error handling
on the fs APIs.

PR-URL: https://github.com/nodejs/node/pull/10739
Ref: https://github.com/nodejs/node/issues/10703
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
2017-02-06 11:03:37 -08:00
James M Snell
5e52a9ac09 Revert "fs: allow WHATWG URL and file: URLs as paths"
This reverts commit 79400bfbfd.

PR-URL: https://github.com/nodejs/node/pull/11155
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: Timothy Gu <timothygu99@gmail.com>
2017-02-03 16:43:18 -08:00
James M Snell
79400bfbfd fs: allow WHATWG URL and file: URLs as paths
Updates the fs module APIs to allow 'file://' URL objects
to be passed as the path.

For example:

```js
const URL = require('url').URL;
const myURL = new URL('file:///C:/path/to/file');
fs.readFile(myURL, (err, data) => {});
```

On Windows, file: URLs with a hostname convert to UNC paths,
while file: URLs with drive letters convert to local absolute
paths:

```
file://hostname/a/b/c => \\hostname\a\b\c
file:///c:/a/b/c => c:\a\b\c
```

On all other platforms, file: URLs with a hostname are unsupported
and will result in a throw:

```
file://hostname/a/b/c => throw!
file:///a/b/c => /a/b/c
```

The documentation for the fs API is intentionally not updated in
this commit because the URL API is still considered experimental
and is not officially documented *at this time*

Note that file: URLs are *required* by spec to always be absolute
paths from the file system root.

This is a semver-major commit because it changes error handling
on the fs APIs.

PR-URL: https://github.com/nodejs/node/pull/10739
Ref: https://github.com/nodejs/node/issues/10703
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
2017-02-03 13:59:06 -08:00
Timothy Gu
0792d452e8 url: fix setting url.search to the empty string
PR-URL: https://github.com/nodejs/node/pull/11105
Fixes: https://github.com/nodejs/node/issues/11101
Fixes: 98bb65f641 "url: improving URLSearchParams"
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2017-02-03 12:29:07 -08:00
James M Snell
c5e9654b5b url: extend url.format to support WHATWG URL
Removes the non-standard options on WHATWG URL toString
and extends the existing url.format() API to support
customizable serialization of the WHATWG URL object.

This does not yet include the documentation updates
because the documentation for the new WHATWG URL object
has not yet landed.

Example:

```js
const url = require('url');
const URL = url.URL;
const myURL = new URL('http://example.org/?a=b#c');
const str = url.format(myURL, {fragment: false, search: false});
console.log(str);
  // Prints: http://example.org/
```

PR-URL: https://github.com/nodejs/node/pull/10857
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Timothy Gu <timothygu99@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
2017-02-02 12:35:55 -08:00
Timothy Gu
cc48f21c83 url: extend URLSearchParams constructor
PR-URL: https://github.com/nodejs/node/pull/11060
Fixes: https://github.com/nodejs/node/issues/10635
Ref: https://github.com/whatwg/url/pull/175
Ref: https://github.com/w3c/web-platform-tests/pull/4523
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2017-01-31 23:51:06 -08:00
Timothy Gu
90c2ac7be3 url: make URLSearchParams/Iterator match spec
PR-URL: https://github.com/nodejs/node/pull/11057
Fixes: https://github.com/nodejs/node/issues/10799
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2017-01-31 09:29:42 -08:00
Timothy Gu
06ecf4dec7 url: define @@toStringTag as a data property
Even though this is not fully Web IDL spec-compliant, it is arguably the
best we can do. Following the spec would mean non-trivial performance
deterioration (10% when parsing a medium-length URL), while the current
getter behavior is not adopted by any implementer, and it causes some
spec ambiguity when the getter is called with !(this instanceof URL).

This commit adopts Chrome's behavior, and is consistent with
ECMAScript-defined classes while providing reasonable behaviors for
corner cases as well. Until the Web IDL spec is changed one way or
another, this is the way to go.

PR-URL: https://github.com/nodejs/node/pull/10906
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2017-01-28 08:27:15 -08:00
Timothy Gu
d65904bc6a url: do not public expose inspect methods on URL
PR-URL: https://github.com/nodejs/node/pull/10906
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2017-01-28 08:27:09 -08:00
Timothy Gu
e71c278288 url: stop exporting originFor()
PR-URL: https://github.com/nodejs/node/pull/10955
Fixes: https://github.com/nodejs/node/issues/10800
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
2017-01-27 07:35:36 -08:00
Timothy Gu
ed0086fb46 url: check forEach callback is a function
The Web IDL spec mandates such a check.

Also make error messages consistent with rest of Node.js and add
additional tests for forEach().

PR-URL: https://github.com/nodejs/node/pull/10905
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
2017-01-23 16:24:33 -08:00
Rich Trott
6b6123cfbe url: refactor lib/internal/url.js
* set an identifier for the separator rather than using multiple
  instances of the same literal
* consistent arrow function body formatting

PR-URL: https://github.com/nodejs/node/pull/10912
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Italo A. Casas <me@italoacasas.com>
Reviewed-By: Timothy Gu <timothygu99@gmail.com>
2017-01-22 21:35:21 -08:00
Timothy Gu
326e967c6b url: export URLSearchParams
Fixes: #10761
2017-01-22 11:27:58 +01:00
Timothy Gu
98bb65f641
url: improving URLSearchParams
- add some benchmarks for URLSearchParams
- change URLSearchParams backing store to an array
- add custom inspection for URLSearchParams and its iterators

PR-URL: https://github.com/nodejs/node/pull/10399
Reviewed-By: James M Snell <jasnell@gmail.com>
2017-01-19 11:33:37 -08:00
James M Snell
0f62ee6963 url: allow use of URL with http.request and https.request
PR-URL: https://github.com/nodejs/node/pull/10638
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Michal Zasso <targos@protonmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2017-01-09 09:28:11 -08:00
James M Snell
5161b00af5 url: change null password handling
Pulls in new URL parsing tests from w3c web-platform-tests and updates
null password handling.

Refs: e001240685
Refs: https://github.com/whatwg/url/pull/186
PR-URL: https://github.com/nodejs/node/pull/10601
Fixes: https://github.com/nodejs/node/issues/10595
Reviewed-By: Michal Zasso <targos@protonmail.com>
Reviewed-By: Italo A. Casas <me@italoacasas.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2017-01-05 20:23:11 -08:00
James M Snell
f5d92f3563 url: set toStringTag for the URL class
PR-URL: https://github.com/nodejs/node/pull/10562
Fixes: https://github.com/nodejs/node/issues/10554
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2017-01-03 15:33:03 -08:00
Joyee Cheung
2177a38178 url: TupleOrigin#toString use unicode by default
See: https://url.spec.whatwg.org/#dom-url-origin

Also moves the tests for origins to the parsing tests
since now URL#origin matches the test cases by default.

PR-URL: https://github.com/nodejs/node/pull/10552
Reviewed-By: James M Snell <jasnell@gmail.com>
2017-01-03 10:37:15 -08:00
James M Snell
ac90d298d4 url: fix accidental filemode change
PR-URL: https://github.com/nodejs/node/pull/10549
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
2017-01-03 10:23:39 -08:00
Michaël Zasso
b465cd07fe url: fix URL query update if searchParams changes
If searchParams becomes empty, query must be set to null.
Add missing update of context flags.

Fixes: https://github.com/nodejs/node/issues/10480
PR-URL: https://github.com/nodejs/node/pull/10486
Reviewed-By: James M Snell <jasnell@gmail.com>
2017-01-01 11:04:55 +01:00
Michaël Zasso
db18dd8356 url: improve spec compliance of WHATWG URL
This patch contains the following changes:

url: make IPv4 parser more spec compliant

* Return int64_t from ParseNumber to prevent overflow for valid big numbers
* Don't throw when there are more than 4 parts (it cannot be an IP
address)
* Correctly interpret the address and don't always throw when there are
numbers > 255

Ref: https://url.spec.whatwg.org/#concept-ipv4-parser
Fixes: https://github.com/nodejs/node/issues/10306

url: percent encode fragment to follow spec change

Ref: https://github.com/whatwg/url/issues/150
Ref: 373dbedbbf

url: fix URL#search setter

The check for empty string must be done before removing the leading '?'.

Ref: https://url.spec.whatwg.org/#dom-url-search

url: set port to null if an empty string is given

This is to follow a spec change.

Ref: https://github.com/whatwg/url/pull/113

url: fix parsing of paths with Windows drive letter

test: update WHATWG URL test fixtures

PR-URL: https://github.com/nodejs/node/pull/10317
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2017-01-01 10:52:35 +01:00
James M Snell
abc1633de6 url: move originFor, domainToAscii and domainToUnicode
Move non-standard methods to `url` module instead of exposing as
static methods on the `URL` object.

PR-URL: https://github.com/nodejs/node/pull/10512
Reviewed-By: Italo A. Casas <me@italoacasas.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
2016-12-30 10:14:10 -08:00
James M Snell
2213f3640a url: performance improvement in URL implementation
Yields about a 25% average performance improvement

PR-URL: https://github.com/nodejs/node/pull/10469
Reviewed-By: Michaël Zasso <targos@protonmail.com>
2016-12-30 09:49:12 -08:00
Joyee Cheung
508d97628b url: make WHATWG URL properties spec compliant
* Set exposed attributes of the interface enumerable
  and configurable, as required by the spec.
  See: https://heycam.github.io/webidl/#es-attributes
* Make sure `URL#searchParams` returns `[[SameObject]]`
* Add the missing `URL#href` setter
* Reorder the properties to match
  https://url.spec.whatwg.org/#api
* Add tests for the ECMAScript property attributes

PR-URL: https://github.com/nodejs/node/pull/10408
Fixes: https://github.com/nodejs/node/issues/10376
Reviewed-By: James M Snell <jasnell@gmail.com>
2016-12-29 15:46:59 -08:00
Jay Brownlee
4f97a146ff
url, test: fix typo in inspect output, add test
In the string returned from URL.inspect there was an extra semicolon
at the end when showHidden === true. The semicolon has been
removed and a test for the inspect function has been added. The test
parses the returned string, validates all of the contained keys/values
and tests logic related to the showHidden option.

PR-URL: https://github.com/nodejs/node/pull/10231
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Evan Lucas <evanlucas@me.com>
2016-12-14 05:20:42 +01:00
joyeecheung
a84017a689 url, test: including base argument in originFor
- Add tests to check if the `originFor` implementation
  for WHATWG url parsing is correnct.
- Fix `originFor` by including a base as argument

PR-URL: https://github.com/nodejs/node/pull/10021
Reviewed-By: James M Snell <jasnell@gmail.com>
2016-12-12 09:54:44 -05:00
Timothy Gu
61d6293033 url: improve URLSearchParams spec compliance
- Make URLSearchParams constructor spec-compliant
- Strip leading `?` in URL#search's setter
- Spec-compliant iterable interface
- More precise handling of update steps as mandated by the spec
- Add class strings to URLSearchParams objects and their prototype
- Make sure `this instanceof URLSearchParams` in methods

Also included are relevant tests from W3C's Web Platform Tests
(https://github.com/w3c/web-platform-tests/tree/master/url).

Fixes: https://github.com/nodejs/node/issues/9302
PR-URL: https://github.com/nodejs/node/pull/9484
Reviewed-By: James M Snell <jasnell@gmail.com>
2016-12-07 12:19:23 -05:00
Safia Abdalla
883173289d
url: add inspect function to TupleOrigin
This adds a simple inspect function the the TupleOrigin class.
This adds tests for the newly added inspect function in the TupleOrigin
class.

PR-URL: https://github.com/nodejs/node/pull/10039
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
2016-12-05 03:12:55 +01:00
James M Snell
4b312387ea url: adding WHATWG URL support
Implements WHATWG URL support. Example:

```
var u = new url.URL('http://example.org');
```

Currently passing all WHATWG url parsing tests and all but two of the
setter tests. The two setter tests are intentionally skipped for now
but will be revisited.

PR-URL: https://github.com/nodejs/node/pull/7448
Reviewed-By: Ilkka Myller <ilkka.myller@nodefield.com>
2016-10-11 12:41:42 -07:00