filmstarts.de - http 403 forbidden

If you need help on how to use the program
Post Reply
klaftan
Posts: 3
Joined: 2016-12-10 18:02:25

filmstarts.de - http 403 forbidden

Post by klaftan »

Hello antp,

first of all I have to say "many thanks" for this great and really helpful piece of software provided by you and all the others.

I am new here because I did not have any problems until now. My database contains about 2500 movies. Most of them were added by using the filmstarts.de script, some of them manually. The script worked perfect until two or three weeks ago.

Now I wanted to add more movies but I always get the error message "HHTP/1.1 403 Forbidden".

Until now I have done the following points:
  • Search for the movie manually ==> worked
    Searched for the bad command: It is the GetPage command
    Tried the GetPage2 command instead ==> no success
    I have created a small script to try these things - here it is:

Code: Select all

program SUKorrektur;

var
	PageText, MovieName, Address, Ref: string;

begin

	Ref := 'http://www.filmstarts.de';
	MovieName := 'Dreckskerle';
	ShowMessage('MovieName = '+Moviename);
	MovieName := UrlEncode(MovieName);
	ShowMessage('MovieName and URL Encode = '+Moviename);
	Address := 'http://www.filmstarts.de/suche/?q='+MovieName;

	ShowMessage('Address = '+Address);
	PageText := GetPage2(Address, Ref);
	ShowMessage(PageText);
  
end.
My question: Is this a problem of the GetPage comman or did they change some access propertied on the filmstarts.de page?

Many thanks for your help

PS: I searched the internet and this forum before posting - I did not find any hint. Unfortunately I cannot read the french postings :mad:
antp
Site Admin
Posts: 9636
Joined: 2002-05-30 10:13:07
Location: Brussels
Contact:

Post by antp »

It seems they use the same anti-bot system that what Allociné started to use recently.
We still haven't found a way to get around that :/
(which in a way is normal, since they try to prevent exactly what we're trying to do...)
klaftan
Posts: 3
Joined: 2016-12-10 18:02:25

Post by klaftan »

Thanks for your answer.
Hopefully you can find a solution.
J
Posts: 224
Joined: 2008-02-17 17:09:26

Post by J »

hm, haven't noticed this yet, but right, filmstarts is part of allocine group so the problem might be the same.

It looks like they use the service of an external company to prevent 'bad bots' from their site.

After playing around changing the useragent in my browser to AMCs useragent, I first got some "you are blocked" pages then magically it works again in the browser and it also worked with AMC for some movies.

And after changing the search link in AMC, more movies are working, but not all.
(Looks like movie titles with one word are working, titles with more words are not - perhaps because of the search string, needs more testing)

has anybody noticed this behaviour (with allocine) too?

PS: I changed the search link from
http://www.filmstarts.de/suche/1/?q=
to
http://www.filmstarts.de/suche/?q=
J
Posts: 224
Joined: 2008-02-17 17:09:26

Post by J »

well, after testing/changing a bit more, it's a little frustrating to see that AMC really seems to be blocked for beeing some kind of 'bad bot' and changing the script is sometimes working, sometimes not.

Because of the use of an external service provider for this, I'm not sure blocking AMC is really intended by allociné group or just some kind of side effect to prevent some other really 'bad bots'. The blocking itself is not very smart, looks like they just testing the user agent somehow.

I've seen you discussed this problem at the allocine script page and as far as I understood (my french is unfortunately at zero level), the solution might be a change of the user agent string for now an ever!? This could be perhaps implemented as an option for another program version. - I tried this, and yes AMC is working perfectly with the script again.

So it looks like there is nothing I can do here for now, but pls. tell me if a scipt based solution is found by someone.

cheers
J.
ILoveYou
Posts: 3
Joined: 2016-12-29 12:43:35

Post by ILoveYou »

---------- SOLUTION ----------

See my solution in french here:
viewtopic.php?t=6370&start=200

To sum up, just add this line in your translation file used after [GetScriptWin]:
http.Request.UserAgent=Mozilla/5.0

Merry Chrismas and Happy New Year!
Long life to Ant Movie Catalog! I Love You!
klaftan
Posts: 3
Joined: 2016-12-10 18:02:25

Post by klaftan »

Hello ILoveYou,

tried your suggested solution - works perfect :grinking:

Many thanks for your hint!
Happy new year to you and all the other ANTP lovers :D
Post Reply