[ antp.be > Forum ]

[ FAQFAQ - SearchSearch - MemberlistMemberlist - UsergroupsUsergroups - ]

[ RegisterRegister - ProfileProfile - Log in to check your private messagesLog in to check your private messages - Log inLog in ]


filmstarts.de - http 403 forbidden

 
Post new topic   Reply to topic    www.antp.be Forum Index -> Ant Movie Catalog > Help
Author Message
klaftan
PostPosted: 2016-12-10 19:23:09    Post subject: filmstarts.de - http 403 forbidden Reply with quote
View user's profile Send private message Hello antp,

first of all I have to say "many thanks" for this great and really helpful piece of software provided by you and all the others.

I am new here because I did not have any problems until now. My database contains about 2500 movies. Most of them were added by using the filmstarts.de script, some of them manually. The script worked perfect until two or three weeks ago.

Now I wanted to add more movies but I always get the error message "HHTP/1.1 403 Forbidden".

Until now I have done the following points:
    Search for the movie manually ==> worked
    Searched for the bad command: It is the GetPage command
    Tried the GetPage2 command instead ==> no success
    I have created a small script to try these things - here it is:


Code:

program SUKorrektur;

var
   PageText, MovieName, Address, Ref: string;

begin

   Ref := 'http://www.filmstarts.de';
   MovieName := 'Dreckskerle';
   ShowMessage('MovieName = '+Moviename);
   MovieName := UrlEncode(MovieName);
   ShowMessage('MovieName and URL Encode = '+Moviename);
   Address := 'http://www.filmstarts.de/suche/?q='+MovieName;

   ShowMessage('Address = '+Address);
   PageText := GetPage2(Address, Ref);
   ShowMessage(PageText);
 
end.


My question: Is this a problem of the GetPage comman or did they change some access propertied on the filmstarts.de page?

Many thanks for your help

PS: I searched the internet and this forum before posting - I did not find any hint. Unfortunately I cannot read the french postings mad
antp
Site Admin
PostPosted: 2016-12-10 21:12:26    Post subject: Reply with quote
View user's profile Send private message Visit poster's website It seems they use the same anti-bot system that what Allociné started to use recently.
We still haven't found a way to get around that oh well
(which in a way is normal, since they try to prevent exactly what we're trying to do...)
klaftan
PostPosted: 2016-12-10 21:45:39    Post subject: Reply with quote
View user's profile Send private message Thanks for your answer.
Hopefully you can find a solution.
J
PostPosted: 2016-12-12 14:48:04    Post subject: Reply with quote
View user's profile Send private message hm, haven't noticed this yet, but right, filmstarts is part of allocine group so the problem might be the same.

It looks like they use the service of an external company to prevent 'bad bots' from their site.

After playing around changing the useragent in my browser to AMCs useragent, I first got some "you are blocked" pages then magically it works again in the browser and it also worked with AMC for some movies.

And after changing the search link in AMC, more movies are working, but not all.
(Looks like movie titles with one word are working, titles with more words are not - perhaps because of the search string, needs more testing)

has anybody noticed this behaviour (with allocine) too?

PS: I changed the search link from
http://www.filmstarts.de/suche/1/?q=
to
http://www.filmstarts.de/suche/?q=
J
PostPosted: 2016-12-13 00:15:40    Post subject: Reply with quote
View user's profile Send private message well, after testing/changing a bit more, it's a little frustrating to see that AMC really seems to be blocked for beeing some kind of 'bad bot' and changing the script is sometimes working, sometimes not.

Because of the use of an external service provider for this, I'm not sure blocking AMC is really intended by allociné group or just some kind of side effect to prevent some other really 'bad bots'. The blocking itself is not very smart, looks like they just testing the user agent somehow.

I've seen you discussed this problem at the allocine script page and as far as I understood (my french is unfortunately at zero level), the solution might be a change of the user agent string for now an ever!? This could be perhaps implemented as an option for another program version. - I tried this, and yes AMC is working perfectly with the script again.

So it looks like there is nothing I can do here for now, but pls. tell me if a scipt based solution is found by someone.

cheers
J.
ILoveYou
PostPosted: 2016-12-29 14:15:44    Post subject: Reply with quote
View user's profile Send private message ---------- SOLUTION ----------

See my solution in french here:
http://forum.antp.be/phpbb2/viewtopic.php?t=6370&start=200

To sum up, just add this line in your translation file used after [GetScriptWin]:
http.Request.UserAgent=Mozilla/5.0

Merry Chrismas and Happy New Year!
Long life to Ant Movie Catalog! I Love You!
klaftan
PostPosted: 2016-12-31 10:19:21    Post subject: Reply with quote
View user's profile Send private message Hello ILoveYou,

tried your suggested solution - works perfect grin kind

Many thanks for your hint!
Happy new year to you and all the other ANTP lovers grin
Display posts from previous:   
View previous topic :: View next topic  
Post new topic   Reply to topic    www.antp.be Forum Index -> Ant Movie Catalog > Help All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2006 phpBB Group
Template made by antp