提取网址的域名

Question

提取网址的域名

另一个解析URL的请求，但我发现了许多不完整或理论上的例子。我想肯定有一些在perl中工作的东西。

我有以下网址：

https://vimdoc.sourceforge.net/htmldoc/pattern.html
http://linksyssmartwifi.com/ui/1.0.1.1001/dynamic/login.html
http://www.catonmat.net/download/perl1line.txt
https://github.com/robbyrussell/oh-my-zsh/wiki/Cheatsheet
https://drive.google.com/drive/u/0/folders/0B5jNDUmF2eUJuSnM
http://www.gnu.org/software/coreutils/manual/coreutils.html
http://www.catonmat.net/download/perl1line.txt
https://feedly.com/i/my
http://vimhelp.appspot.com/
https://git-scm.com/doc
https://read.amazon.com/
https://github.com/netsamir/following
https://scotch.io/
https://servicios.dgi.gub.uy/
https://sourcemaking.com/
https://stackedit.io/editor
https://stripe.com/be
https://toolbelt.heroku.com/
https://training.github.com/
https://vimeo.com/54505525
https://vimeo.com/tag:drew+neil
https://web.whatsapp.com/
https://www.ctan.org/
https://www.eff.org/
https://www.mybeluga.com/
https://www.solveforx.com/
https://www.symynd.com/
https://www.symynd.com/#
https://www.tizen.org/
http://workforall.net/CDS-Credit-default-Swaps.html#Credit_Default_Swaps_CDS

尝试仅提取域名。例如：

linksyssmartwifi.com
amazon.com
github.com

我曾尝试使用Perl和Vim，但无法完成任务。我最好的近似值如下

 perl -pe 's!(^https?\://.*[\.](.+\..+?)/.*$)!$1 -- [$2] !g' all_urls_sorted.txt

其中一些被正确解析（见[]），其他不是：

   https://sites.google.com/site/steveyegge2/singleton-considered-stupid -- [google.com] 
https://sourcemaking.com/
https://stackedit.io/editor
https://stripe.com/be
https://toolbelt.heroku.com/ -- [heroku.com] 
https://training.github.com/ -- [github.com] 
https://vimeo.com/54505525
https://vimeo.com/tag:drew+neil
https://web.whatsapp.com/ -- [whatsapp.com] 
https://wiki.haskell.org/GHC -- [haskell.org]

正如我的测试显示的那样，直接从//（以https？：//开头）的URL被排除在外。

如果你知道如何解决这个问题，我会非常高兴。

感谢

Answer 1

使用URI模块：

#!/usr/bin/env perl

use strict;
use warnings;
use v5.10;

use URI;

while (<DATA>) {
    chomp;
    my $uri = URI->new($_);
    my $host = $uri->host;
    my ($domain) = $host =~ m/([^.]+\.[^.]+$)/;
    say $domain;
}

__DATA__
https://vimdoc.sourceforge.net/htmldoc/pattern.html
http://linksyssmartwifi.com/ui/1.0.1.1001/dynamic/login.html
http://www.catonmat.net/download/perl1line.txt
https://github.com/robbyrussell/oh-my-zsh/wiki/Cheatsheet
https://drive.google.com/drive/u/0/folders/0B5jNDUmF2eUJuSnM
http://www.gnu.org/software/coreutils/manual/coreutils.html
http://www.catonmat.net/download/perl1line.txt
https://feedly.com/i/my
http://vimhelp.appspot.com/
https://git-scm.com/doc
https://read.amazon.com/
https://github.com/netsamir/following
https://scotch.io/
https://servicios.dgi.gub.uy/
https://sourcemaking.com/
https://stackedit.io/editor
https://stripe.com/be
https://toolbelt.heroku.com/
https://training.github.com/
https://vimeo.com/54505525
https://vimeo.com/tag:drew+neil
https://web.whatsapp.com/
https://www.ctan.org/
https://www.eff.org/
https://www.mybeluga.com/
https://www.solveforx.com/
https://www.symynd.com/
https://www.symynd.com/#
https://www.tizen.org/
http://workforall.net/CDS-Credit-default-Swaps.html#Credit_Default_Swaps_CDS

输出：

sourceforge.net
linksyssmartwifi.com
catonmat.net
github.com
google.com
gnu.org
catonmat.net
feedly.com
appspot.com
git-scm.com
amazon.com
github.com
scotch.io
gub.uy
sourcemaking.com
stackedit.io
stripe.com
heroku.com
github.com
vimeo.com
vimeo.com
whatsapp.com
ctan.org
eff.org
mybeluga.com
solveforx.com
symynd.com
symynd.com
tizen.org
workforall.net

Answer 2

我的最佳近似值是URI::URL：

foreach my $uri (@filecontents) {
    my $uriobj = URL::URL->new($uri);
    my $host = $uriobj -> host;
    my @parts = split /\./, $host;
    print "$uri -- $parts[-2]$parts[-1]\n";
}

希望有所帮助。

Answer 3

正则表达式解决方案是：

//(?:[^./]+[.])*([^/.]+[.][^/.]+)/

如果尾部斜杠是可选的，只需添加?：

//(?:[^./]+[.])*([^/.]+[.][^/.]+)/?

这应该与全局修饰符和/以外的分隔符一起使用。

基本上，它在//和下一个/之间寻找。

如果有任何额外的子域，它们将被(?:[^./]+[.])*捕获。主域名将属于捕获组([^/.]+[.][^/.]+)。

Perl：提取域名

提取网址的域名

3 个答案: