I can't use php curl or php get file_get_contents to the website "https://vietnamairlines.com""
I don't know the reason why I can't do curl either
file_get_contents go to the website "https://www.vietnamairlines.com/" using the linode server. The commands still work under localhost, but when I execute it on my server it returns "502 bad gateway". Hope you guys can help me to solve this problem. Thank you !
So basically what is happening here is when you (or I) try to visit the airline website from our local computers it works fine. When trying to curl the same website from the Linode server, you are experiencing a 502 bad gateway error. This is likely because many (most) airlines block access from cloud servers and proxy IP addresses in efforts to prevent price gouging (AKA ticket scalping).
This is exemplified in similarly worded Stackexchange posts:
The only potential solution here for your curl dilemma is an IP swap, which you would have to open a support ticket to achieve, but this is not guaranteed to resolve your issues, since Linode is not privy to which IPs are blocked by this specific airline company.
Hope that helps!
Also make sure you send a browser user agent otherwise it’s fairly obvious it’s not a client browser!
What you are trying to do is use a technique that spammers/scammers/etc have used for years to acquire email addresses and other personal info to use for nefarious purposes. It's not just airlines that block the technique, a great number of web-developers/-masters do that. It's a common technique in the web design biz for creators to protect the intellectual property of their design(s) for a particular customer.
It's also one of the services of organizations like Cloudflare and Akamai.
I run a site for a fraternal organization and I won't let 'bots download the site with curl or file_get_contents either…even though all the email addresses, phone numbers and other info I consider "personal" are all encrypted/encoded in the pages.
You may think your motives are innocent enough but, to the owner/creator of the page(s) you're trying to download, you're a thief.
FWIW, when I tried this, I got an expired certificate error.
stevewi@dave:~$ curl https://vietnamairlines.com curl: (60) SSL certificate problem: certificate has expired More details here: https://curl.haxx.se/docs/sslcerts.html curl failed to verify the legitimacy of the server and therefore could not establish a secure connection to it. To learn more about this situation and how to fix it, please visit the web page mentioned above.
I got the same thing with Safari/Chrome on my laptop.