Page 1 of 1

IMDb script - add writer, composer

Posted: 2014-05-08 17:49:57
by elman
Hi guys,

now that 4.2.0 is officially out, I'd like to ask you to do couple of fixes for IMDb script, if possible:

1. Add Writer, so that it automatically saved during import
2. Add Composer, so that it automatically saved during import
3. Fix IMDb Trivia, so that strings like "X of X found this interesting" or "Is this interesting?" is removed when saving trivia
4. Round Rating when "Do not display decimals" is used, so that movie with rating 3,9 won't be saved as 3.
5. Certification should be put to the correct field, since we have one now.

Thanks a lot,
Elman

Posted: 2014-10-26 14:00:48
by elman
I got tired of waiting, so I made most of the changes myself. I will share it, however one thing remains to be done (trivia problem). Could anyone share the documentation for regexps? And what language are these scripts written in anyway? Looks like Pascal, but I'm note sure...

Posted: 2014-10-27 09:52:28
by antp
The language used is indeed very close to Pascal/Delphi
(it uses mostly the same syntax and functions, the differences are minor)
In the help file, in Technical info -> Script files creation, you can find the detail of functions added to the script engine especially for AMC, including the regex functions.

Posted: 2014-10-27 20:07:15
by elman
Ah, never thought of checking help file :)
Thanks. Will read it now.

Posted: 2014-10-28 22:47:39
by elman
I finished the script. It will only work with AMC 4.2.0 and hopefully I didn't break any functionality.

Change list:
- Writer is automatically imported
- Composer is automatically imported
- Fixed IMDb Trivia, so that strings like "X of X found this interesting" or "Is this interesting?" is removed
- Added Rating rounding so that movie with rating 3,9 won't be saved as 3 but rather 4. For this you will have to set script option RoundRating to 1.
- Certification is put to the correct field
- Removed empty space after Original title if translation exists
- Removed empty space after description if IMDB trivia exists
- All directors are now imported
- Removed extra line if CommentType=0

Code: Select all

(***************************************************

Ant Movie Catalog importation script
www.antp.be/software/moviecatalog/

[Infos]
Authors=Antoine Potten, KaraGarga, baffab, Thermal Ions, bad4u, Sancho, Joe, cage, Elman
Title=IMDB
Description=Import data & picture from IMDB (optional image from Amazon)
Site=us.imdb.com
Language=EN
Version=3.86
Requires=4.2.0
Comments=Based on the script made for version 3.3/3.4 by Antoine Potten, Danny Falkov, Kai Blankenhorn, lboregard, Ork, Trekkie, Youri Heijnen
License=This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
GetInfo=1
RequiresMovies=1

[Options]
ActorsLayout=2|0|0=Only actor names, separated by commas|1=Only actor names, separated by linebreaks|2=Actors names with character names between parenthesis separated by commas|3=Actors names with character names between parenthesis separated by linebreaks|4=Actor names like on IMDB page, with "...." and separated by linebreaks
AllActors=1|0|0=Only import actors from main page (does not work if you import producer name)|1=Import all possible actors|2=Import 10 first actors (does not work with ActorLayout=4)
AspectRatio=1|1|0=Do not import picture aspect ratio|1=Import picture aspect ratio to video format field|2=Import picture aspect ratio to resolution field
Awards=2|0|0=Do not import awards|1=Import awards to Description field, after the summary|2=Import awards to Comments field, after comments
BatchMode=0|0|0=Normal working mode, prompts user when needed|1=Does not display any window, takes the first movie found|2=Same as 1, but it uses the URL field if available to update movie information
CommentType=0|0|0=Standard Type (Only one comment from main page)|1=Detailed Type (10 most useful comments from comments page)|2=No user comment, clear current field contents|3=No user comment, keep current field contents (may cause problem for multiple imports on the same movie if other options append text to the comment field)
ConvertToASCII=0|0|0=Do not change special characters and accents.|1=Replace special characters and accents by basic ASCII characters
DescriptionSelection=0|0|0=Take the short summary, from main page (faster)|1=Show a list of available summaries|2=Take the longest summary
EpisodeTitleSearch=0|0|0=Use "Find more" button on results list for next result pages if available (default)|1=Use "Find more" button on results list for episode title search instead
GetTagline=1|1|0=Do not get tagline|1=Put it in Description field, before the summary|2=Put it in the Comment field, before the comments
HideAkaTitles=0|0|0=Show 'aka' titles on result list (= other titles for the film)|1=Hide 'aka' titles on result list
ImageKind=3|3|0=No image|1=IMDB small image, from the main movie page, no image if none available|2=IMDB small image, from the main movie page, "No Poster Available" if none available|3=IMDB large image, from link on main movie page, else small image, else "No Poster Available"|4=Cover from "Merchandising Link" page, else "DVD detail" page, else small image|5=Cover from "DVD detail" page, else "Merchandising Link", else small image
MPAA=2|0|0=Do not import MPAA rating|1=Import MPAA rating to MediaType|2=Append MPAA rating and info to Comments
MultipleValuesAudioFormat=1|1|0=Only take first value for Audio Format|1=Take full list, separated by commas|2=Take full list, separated by slashes|3=Do not import Audio Format / Sound Mix
MultipleValuesCategory=1|1|0=Only take first value for Category|1=Take full list, separated by commas|2=Take full list, separated by slashes|3=Do not import Category
MultipleValuesCountry=1|1|0=Only take first value for Country|1=Take full list, separated by commas|2=Take full list, separated by slashes|3=Do not import Country
MultipleValuesLanguages=1|1|0=Only take first value for Languages|1=Take full list, separated by commas|2=Take full list, separated by slashes|3=Do not import Languages
PopularSearches=1|1|0=Do not use the popular searches page, directly show full search results|1=Show popular searches first, I'll click on "Find more" if needed (much faster)
RoundRating=0|0|0=Do not round Rating|1=Round Rating to whole number
Trivia=0|0|0=Do not import trivia|1=Import short trivia to Description field, after the summary|2=Import short trivia to Comments field, after the comments|3=Import full trivia to Description field, after the summary|4=Import full trivia to Comments field, after the comments
UserRatings=0|0|0=Import value to ratings field only (default)|1=Import value and number of votes to Media Type field|2=Import value and number of votes to comments field

[Parameters]

***************************************************)

program IMDB;

uses
  StringUtils1;

  // ***** Manually set UserCountry to your required Classification Country below *****
const
  //UserCountry = '';
  { Delete the line above and remove the "//" in front of one the
    following lines, or add your country if it is not listed }
  UserCountry = 'USA';
  //UserCountry = 'Canada';
  //UserCountry = 'Mexico';
  //UserCountry = 'Brazil';
  //UserCountry = 'Argentina';
  //UserCountry = 'Australia';
  //UserCountry = 'India';
  //UserCountry = 'Italy';
  //UserCountry = 'Spain';
  //UserCountry = 'Portugal';
  //UserCountry = 'France';
  //UserCountry = 'Germany';
  //UserCountry = 'Netherlands';
  //UserCountry = 'UK';
  //UserCountry = 'Ireland';
  //UserCountry = 'Finland';
  //UserCountry = 'Norway';
  //UserCountry = 'Sweden';
  //UserCountry = 'Switzerland';

  PopularTitleSearchURL =  'http://www.imdb.com/find?s=tt&q=';
  FullTitleSearchURL = 'http://www.imdb.com/find?s=tt&exact=true&q=';
  EpisodeTitleSearchURL = 'http://www.imdb.com/find?s=ep&q=';
  
var
  MovieName: string;
  MovieURL: string;
  MovieNumber: string;
  UpdateFile: TStringList;
  
function ConvertToASCII(AText: string): string;
begin
  Result := AText;
  if GetOption('ConvertToASCII') = 1 then
    Result := Cp1252ToASCII(AText);
end;

// ***** analyzes IMDB's results page that asks to select a movie from a list *****

procedure AnalyzeResultsPage(Address: string);
var
  PageText: string;
  Value: string;
begin
  PageText := ConvertToASCII(GetPage(Address));
  if pos('<title>Find - IMDb', PageText) = 0 then
    begin
      AnalyzeMoviePage(PageText)
    end else
    begin
      if Pos('<b>No Matches.</b>', PageText) > 0 then
        begin
          if GetOption('BatchMode') = 0 then
            ShowMessage('No movie found for this search.');
          Exit;
        end;
      if GetOption('BatchMode') = 0 then
        begin
          PickTreeClear;
          Value := TextBetween(PageText, '<h1 class="findHeader">Displaying', '</h1>');
          if Value <> '' then
          begin
            HTMLRemoveTags(Value);
            HTMLDecode(Value);
            PickTreeAdd(UTF8Decode(Value), '');
          end;
          Value := TextBetween(PageText, '<table class="findList">', '</table>');
          PageText := RemainingText;
          AddMovieTitles(Value);
          {
          Value := TextBefore(PageText, '"><b>more titles</b></a>', '<a href="');
          if Value <> '' then
            PickTreeMoreLink('http://www.imdb.com' + Value);
          if GetOption('EpisodeTitleSearch') > 0 then
            PickTreeMoreLink(EpisodeTitleSearchURL + UrlEncode(MovieName));
          }
          if PickTreeExec(Address) then
            AnalyzeResultsPage(Address);
        end
      else
        begin
          Value := TextBetween(PageText, '<td class="primary_photo">', '<img src'); 
          if Value <> '' then
			AnalyzeResultsPage('http://www.imdb.com' + TextBetween(Value, '<a href="', '"'));
        end;
    end;
end;

// ***** adds the movie titles found on IMDB's results page *****

function AddMovieTitles(List: string): Boolean;
var
  Value: string;
  Address: string;
begin
  Result := False;
  Value := TextBetween(List, '<td class="result_text">', '</td>');
  if GetOption('HideAkaTitles') = 1 then
    Value := StringReplace(Value, TextAfter(Value, '<br/>aka'), '')
  else
    Value := StringReplace(Value, 'aka', ' | aka');
  List := RemainingText;
  while Value <> '' do
  begin
    Address := TextBetween(Value, '<a href="/title/tt', '/');
    Address := Address + '/combined';
    HTMLRemoveTags(Value);
    HTMLDecode(Value);
    PickTreeAdd(UTF8Decode(Value), 'http://www.imdb.com/title/tt' + Address);
    Result := True;
    Value := TextBetween(List, '<td class="result_text">', '</td>');
    if GetOption('HideAkaTitles') = 1 then
      Value := StringReplace(Value, TextAfter(Value, '<br/>aka'), '')
    else
      Value := StringReplace(Value, 'aka', ' | aka');
     List := RemainingText;
  end;
end;

// ***** analyzes the page containing movie information *****

procedure AnalyzeMoviePage(PageText: string);
var
  Value, Value2, Value3, FullValue, originalTitle: string;
  p, Count: Integer;
begin
  MovieNumber := TextBetween(PageText, '<input type="hidden" name="auto" value="legacy/title/tt', '/');
  if MovieNumber = '' then
    MovieNumber := TextBetween(PageText, '<input type="hidden" name="auto" value="legacy/title/tt', '/combined"');
  if MovieNumber = '' then
    MovieNumber := TextBetween(PageText, '<link rel="canonical" href="http://www.imdb.com/title/tt', '/');
// #########
//  if ((GetOption('AllActors') = 1) or (GetOption('Producer') = 1)) and (Pos('<div id="tn15" class="maindetails">', PageText) > 0) then
//    PageText := ConvertToASCII(GetPage('http://www.imdb.com/title/tt' + MovieNumber + '/combined'));
  if Pos('/combined"', TextBetween(PageText, '<link rel="canonical"', '/>')) = 0 then
    PageText := ConvertToASCII(GetPage('http://www.imdb.com/title/tt' + MovieNumber + '/combined'));
// #########
  MovieURL := 'http://imdb.com/title/tt' + MovieNumber;
  // URL
  if CanSetField(fieldURL) then
    SetField(fieldURL, MovieURL);
  // OriginalTitle & Year
  if CanSetField(fieldOriginalTitle) or CanSetField(fieldYear) then
  begin
    originalTitle := TextBefore(PageText, '<i>(original title)', '<span class="title-extra">');
    HTMLDecode(originalTitle);
    Value := TextBetween(PageText, '<title>', '</title>');
   // showmessage(originalTitle);
   // showmessage(value);
    if (originalTitle <> '') and CanSetField(fieldOriginalTitle) then
      SetField(fieldOriginalTitle, FullTrim(originalTitle));
    p := Pos(' (1', Value);
    if p = 0 then
      p := Pos(' (2', Value);
    Value2 := Copy(Value, 0, p-1);
    Value := Copy(Value, p+2, Length(Value));
    HTMLDecode(Value2);
    if (originalTitle = '') and CanSetField(fieldOriginalTitle) then
      SetField(fieldOriginalTitle, FullTrim(Value2));
    if Pos('/', Value) > 0 then
      Value2 := TextBefore(Value, '/', '')
    else
      Value2 := TextBefore(Value, ')', '');
    if CanSetField(fieldYear) then
      SetField(fieldYear, Value2);
  end;
  // Picture
  if CanSetPicture then
  begin
    case GetOption('ImageKind') of
      2:  if not ImportSmallPicture(PageText) then
              ImportPictureNotAvailable(PageText);
      3:  if not ImportLargePicture(PageText) then
              if not ImportSmallPicture(PageText) then
                ImportPictureNotAvailable(PageText);
      4:  if not ImportMerchandisingPicture then
              if not ImportDvdDetailsPicture then
                ImportSmallPicture(PageText);
      5:  if not ImportDvdDetailsPicture then
              if not ImportMerchandisingPicture then
                ImportSmallPicture(PageText);
    else
      ImportSmallPicture(PageText);
    end;
  end;

  // Director
  if CanSetField(fieldDirector) then
  begin
    Value := TextBetween(PageText, 'Directed by</a></h5>', '</table>');
    FullValue := '';
    Value2 := TextBetween(Value, '<a href="/name/', '</a>');
    while Value2 <> '' do
    begin
      Value := RemainingText;
      Value2 := TextAfter(Value2, '">');
      if FullValue <> '' then
        FullValue := FullValue + ', ';
      FullValue := FullValue + Value2;
      Value2 := TextBetween(Value, '<a href="/name/', '</a>');
    end;
    HTMLDecode(FullValue);
    SetField(fieldDirector, FullValue);
  end;

  // Actors
  if CanSetField(fieldActors) then
  begin
    Value := FullTrim(TextBetween(PageText, '<table class="cast">', '</table>'));
    if Value <> '' then
    begin
      FullValue := '';
      Count := 0;
      case GetOption('ActorsLayout') of
        0, 1:
          while Pos('<tr', Value) > 0 do
          begin
            Value2 := TextBetween(Value, '<tr', '</tr>');
            Value := RemainingText;
            if Pos('rest of cast', Value2) > 0 then
              Continue;
            if FullValue <> '' then
              FullValue := FullValue + #13#10;
            TextBefore(Value2, '</td>', '');
            Value2 := FullTrim(TextBetween(Value2, '<td class="nm">', '</td>'));
            HTMLRemoveTags(Value2);
            if Value2 <> '' then
            begin
              FullValue := FullValue + Value2;
              Count := Count + 1;
            end;
// ###
            if (Count = 15) and (GetOption('AllActors') = 0) then
              Break;
// ###
            if (Count = 10) and (GetOption('AllActors') = 2) then
              Break;
          end;
        2, 3:
          while Pos('<tr', Value) > 0 do
          begin
            Value2 := TextBetween(Value, '<tr', '</tr>');
            Value := RemainingText;
            if Pos('rest of cast', Value2) > 0 then
              Continue;
            if FullValue <> '' then
              FullValue := FullValue + #13#10;
            TextBefore(Value2, '</td>', '');
            Value2 := FullTrim(TextBetween(Value2, '<td class="nm">', '</td>'));
            HTMLRemoveTags(Value2);
            if Value2 <> '' then
            begin
              FullValue := FullValue + Value2;
              Value2 := FullTrim(TextBetween(RemainingText, '"char">', '</td>'));
              if Value2 <> '' then
                FullValue := FullValue + ' (as ' + Value2 + ')';
              Count := Count + 1;
// ###
              if (Count = 15) and (GetOption('AllActors') = 0) then
                Break;
// ###
              if (Count = 10) and (GetOption('AllActors') = 2) then
                Break;
            end;
          end;
        4:
          begin
            FullValue := Value;
            FullValue := StringReplace(FullValue, ' <tr><td align="center" colspan="4"><small>rest of cast listed alphabetically:</small></td></tr>', '');
            FullValue := StringReplace(FullValue, '>  <', '><');
            FullValue := StringReplace(FullValue, '</tr>', #13#10);
          end;
      end;
      HTMLRemoveTags(FullValue);
      HTMLDecode(FullValue);
      case GetOption('ActorsLayout') of
        0, 2:
          FullValue := StringReplace(FullValue, #13#10, ', ');
      end;
      SetField(fieldActors, FullValue);
    end;
  end;

  // Composer
  if CanSetField(fieldComposer) then
  begin    
    Value := TextBetween(PageText, 'Music by</a></h5>', '</table>');
    FullValue := '';
    Value2 := TextBetween(Value, '<a href="/name/', '</a>');
    while Value2 <> '' do
    begin
      Value := RemainingText;
      Value2 := TextAfter(Value2, '">');
      if FullValue <> '' then
        FullValue := FullValue + ', ';
      FullValue := FullValue + Value2;
      Value2 := TextBetween(Value, '<a href="/name/', '</a>');
    end;
    HTMLDecode(FullValue);
    SetField(fieldComposer, FullValue);
  end;

  //Country
  if CanSetField(fieldCountry) then
  begin
    SetField(fieldCountry, ImportList(PageText, GetOption('MultipleValuesCountry'), '/country/'));
  end;
  //Category
  if CanSetField(fieldCategory) then
  begin
    SetField(fieldCategory, ImportList(PageText, GetOption('MultipleValuesCategory'), '/Genres/'));
  end;
  // Language
  if CanSetField(fieldLanguages) then
  begin
    SetField(fieldLanguages, ImportList(PageText, GetOption('MultipleValuesLanguages'), '/language/'));
  end;
  // Audio Format
  if CanSetField(fieldAudioFormat) then
  begin
    SetField(fieldAudioFormat, ImportList(PageText, GetOption('MultipleValuesAudioFormat'), '/search/title?sound_mixes='));
  end;
  // Aspect Ratio
  begin
    Value := '';
    Value := TextBetween(PageText, '<h5>Aspect Ratio:</h5>', '</div>');
    if Pos('<p>', Value) > 0 then
      Value := TextBetween(Value, '<p>', '</p>');
    if Pos('<a ', Value) > 0 then
      Value := TextBefore(Value, '<a ', '');
    HTMLRemoveTags(Value);
    HTMLDecode(Value);
    Value := FullTrim(Value);
    Value := StringReplace(Value, ', more', '');
    Value := StringReplace(Value, ', (more)', '');
    Value := StringReplace(Value, ' more', '');
    if (CanSetField(fieldVideoFormat)) and (GetOption('AspectRatio') = 1) then
      SetField(fieldVideoFormat, Value);
    if (CanSetField(fieldResolution)) and (GetOption('AspectRatio') = 2) then
      SetField(fieldResolution, Value);
  end;
  // Description
  if CanSetField(fieldDescription) then
  begin
    Value := TextBetween(PageText, '<h5>Plot:</h5>', '</div>');
    if Value = '' then
      Value := TextBetween(PageText, '<h5>Plot Summary:</h5>', '</div>');
    if Pos('<p>', Value) > 0 then
      Value := TextBetween(Value, '<p>', '</p>');
    Value := StringReplace(Value, '<div class="info-content">', '');
    Value := StringReplace(Value, Textbetween(Value, '| <a class="tn15more inline" href="synopsis">', '</a>'), '');
    Value := StringReplace(Value, Textbetween(Value, '<a class="tn15more inline" href="synopsis">', '</a>'), '');
    Value := StringReplace(Value, '| <a class="tn15more inline" href="synopsis"></a>', '');
    Value := StringReplace(Value, '<a class="tn15more inline" href="synopsis"></a>', '');
    Value := StringReplace(Value, '»', '');
    if (Value = #13#10 + #13#10) or (Value = #13#10) then
      Value := '';
    if (GetOption('DescriptionSelection') = 0) and (Pos('<a class="tn15more inline"', Value) > 0) then
      Value := TextAfter(Value, #13#10);
    if Value <> '' then
      SetField(fieldDescription, ImportSummary(Value));
  end;
  // Length
  if CanSetField(fieldLength) then
  begin
    Value := TextBetween(PageText, '<h5>Runtime:</h5>', '</div>');
    if Pos('<p>', Value) > 0 then
      Value := TextBetween(Value, '<p>', '</p>');
    Value := TextBefore(Value, ' min', '');
    HTMLRemoveTags(Value);
    Value := FullTrim(Value);
    if Value <> '' then
    begin
      if Pos(':', Value) > 0 then
        SetField(fieldLength, TextAfter(Value, ':'))
      else
        SetField(fieldLength, Value);
    end;
  end;

  // Producer
  if CanSetField(fieldProducer) then
  begin
    Value := TextBetween(PageText, 'Produced by</a></h5>', '</table>');
    FullValue := '';
    Value2 := TextBetween(Value, '<a href="/name/', '</a>');
    while Value2 <> '' do
    begin
      Value := RemainingText;
      Value2 := TextAfter(Value2, '">');
      if FullValue <> '' then
        FullValue := FullValue + ', ';
      FullValue := FullValue + Value2;
      Value2 := TextBetween(Value, '<a href="/name/', '</a>');
    end;
    HTMLDecode(FullValue);
    SetField(fieldProducer, FullValue);
  end;

  // Writer
  if CanSetField(fieldWriter) then
  begin    
    Value := TextBetween(PageText, 'Writing credits</a></h5>', '</table>');
    FullValue := '';
    Value2 := TextBetween(Value, '<a href="/name/', '</a>');
    while Value2 <> '' do
    begin
      Value := RemainingText;
      Value2 := TextAfter(Value2, '">');
      if FullValue <> '' then
        FullValue := FullValue + ', ';
      FullValue := FullValue + Value2;
      Value2 := TextBetween(Value, '<a href="/name/', '</a>');
    end;
    HTMLDecode(FullValue);
    SetField(fieldWriter, FullValue);
  end;

  // AKA Name
  if CanSetField(fieldTranslatedTitle) then
  begin
    FullValue := ConvertToASCII(GetPage(MovieURL+'/releaseinfo#akas'));
    FullValue := TextBetween(FullValue, '<a id="akas" name="akas">', '</table>');
    FullValue := RemoveSpaces(FullValue, True);
    FullValue := StringReplace(FullValue, #13, '');
    FullValue := StringReplace(FullValue, #10, '');
    FullValue := StringReplace(FullValue, '> <', '><');
    FullValue := StringReplace(FullValue, '<tr class="even">', '<tr>');
    FullValue := StringReplace(FullValue, '<tr class="odd">', '<tr>');
    if RegExprSetExec('</td>\s*<tr>', FullValue) then
      // removing the original title
      FullValue := '<tr>'+TextAfter(FullValue, '</td><tr>');

    while fullvalue <> '' do
    begin
  //   showMessage('Analysing: '+FullValue);
    // each line is a pair of  translated title and country
    // format: <tr><td>country</td><td>Translated title</td><tr>
      value2 := TextBetween(FullValue, '<tr>', '</tr>');
    // showMessage('Line: '+value2);
      value := TextBetween(Value2, '<td>', '</td>');
      value2 := TextAfter(Value2, '</td>');
      value2 := TextBetween(Value2, '<td>', '</td>');
      HTMLDecode(value2);
   //  showMessage('Translated: '+value+' into '+value2);
      if pos(UserCountry, value) > 0 then
      begin
        HTMLDecode(Value2);
        SetField(fieldTranslatedTitle, FullTrim(UTF8Decode(Value2)));
        FullValue := '';
      end
      else  FullValue := TextAfter(FullValue, '</tr>');
    end;
 {
    if Value <> '' then
    begin
      Value := StringReplace(Value, 'See more', '');
      Value := StringReplace(Value, '»', '');
      Value := FullTrim(StringReplace(Value, '<br>', ', '));
      HTMLRemoveTags(Value);
      HTMLDecode(Value);
      Value := FullTrim(Value);
      if Value <> '' then
        if StrGet(Value, Length(Value)) = ',' then
          Delete(Value, Length(Value), 1);
      SetField(fieldTranslatedTitle, Value)
   end;
   }
  end;
  // Comments
  if CanSetField(fieldComments) then
  begin
    if (GetOption('CommentType') = 1) then
    begin
      //Value := TextAfter(PageText,'/rg/title-nav-item/usercomments/');
      //if (Value <> '') or (Pos('href="usercomments" class="link"', PageText) > 0) then
      begin
        Value2 := '';
        FullValue := ConvertToASCII(GetPage(MovieURL+'/usercomments'));
        FullValue := TextAfter(FullValue, 'review useful:</small><br>');
        while FullValue <> '' do
        begin
          Value := TextBetween(FullValue, '<b>', '<div');
          //Value2 := Value2 + #13#10 + #13#10 + TextBefore(Value, '</b>', '');         
          TextBefore(Value, '</b>', '');         
          Value2 := Value2 + #13#10 + #13#10 + TextBetween(Value, '">', '</a>');
          Value := RemainingText;

          Value2 := Value2 + #32+ TextBetween(Value, '<small>', '</small>');
          Value := RemainingText;


          Value2 := Value2 + #32 + 'said on ' + TextBetween(Value, '<small>', '</small>');
          Value := RemainingText;
          Value := TextBetween(Value, 'comments">', '<br>');
          HtmlRemoveTags(Value);
          Value2 := Value2 + #13#10 + Value;
          Value := RemainingText;
          Value := TextBetween(Value, '<p>' + #13#10, #13#10 + '</p>');
          Value := StringReplace(Value, #13#10, ' ');
          Value := StringReplace(Value, '<br><br>', #13#10);
          Value := StringReplace(Value, '<br>', #13#10);
          HtmlDecode(Value);
          Value2 := Value2 + #13#10 + FullTrim(Value);
          FullValue := TextAfter(FullValue, 'review useful:</small><br>');
        end;
        HTMLRemoveTags(Value2);
        HTMLDecode(Value2);
        SetField(fieldComments, 'USER COMMENTS:' + Value2 + #13#10);
      end;
    end
    else
    if (GetOption('CommentType') = 0) then
    begin
      Value := TextAfter(PageText, '/comments">');
      if Value <> '' then
      begin
        Value := TextBetween(Value, '<p>', '</p>');
        Value := StringReplace(Value, #13#10, ' ');
        Value := StringReplace(Value, '<br>', #13#10);
        HTMLRemoveTags(Value);
        HTMLDecode(Value);
        Value := FullTrim(Value);
        while Pos('  ', Value) > 0 do
          Value := StringReplace(Value, '  ', ' ');
        while Pos(#13#10, Value) = 1 do
          Delete(Value, 1, 2);
        SetField(fieldComments, Value);
      end;
    end
    else
    if (GetOption('CommentType') = 2) then
      SetField(fieldComments, '');
  end;
  // TagLine
  if GetOption('GetTagline') > 0 then
  begin
    Value := TextBetween(PageText, '<h5>Tagline:</h5>', '</div>');
    if Pos('<p>', Value) > 0 then
      Value := TextBetween(Value, '<p>', '</p>');
    if Pos('<a', Value) > 0 then
      Value := TextBefore(Value, '<a', '');
    Value := TextAfter(Value, #13#10);
    HTMLRemoveTags(Value);
    HTMLDecode(Value);
    Value := FullTrim(Value);
    if Value <> '' then
    begin
      if StrGet(Value, 1) <> '"' then
        Value := '"' + Value + '"';
      case GetOption('GetTagline') of
        1:
          begin
            if GetField(fieldDescription) <> '' then
              Value := Value + #13#10 + #13#10 + GetField(fieldDescription);
            SetField(fieldDescription, Value);
          end;
        2:
          begin
            if GetField(fieldComments) <> '' then
              Value := Value + #13#10 + #13#10 + GetField(fieldComments);
            SetField(fieldComments, Value);
          end;
      end;
    end;
  end;
  // Trivia
  if GetOption('Trivia') > 0 then
  begin
    sleep(50);
    Value := MovieUrl;
    FullValue := ConvertToASCII(GetPage(Value+'/trivia'));
    case GetOption('Trivia') of
      1,2:
        Value := TextBetween(FullValue, '<div class="sodatext">', '</div>');
      3,4:
        Value := TextBetween(FullValue, '<div class="list">', '<div id="sidebar">');
    end;
    if Value <> '' then
    begin
      Value := StringReplace(Value, #13#10, '');
      while Pos('  ', Value) > 0 do
        Value := StringReplace(Value, '  ', '');
      while Pos('<span class="linksoda">', Value) > 0 do
      begin
        Value := StringReplace(Value, TextBetween(Value, '<span class="linksoda">', '</div>'), '');
        Value := StringReplace(Value, '<span class="linksoda"></div>', '');
      end;
      Value := StringReplace(Value, 'Link this trivia', '');
      Value := StringReplace(Value, '<div class="sodatext">', #13#10 + '- ');
      Value := StringReplace(Value, TextBetween(Value, '<script type="text/javascript">', '</script>'), '');
      HTMLRemoveTags(Value);
      HTMLDecode(Value);
      Value := RegExprSetReplace('\s[\d,]+\s+of\s+[\d,]+\sfound\sthis\sinterestingInteresting\?YesNo\|', Value, '', false);
      Value := RegExprSetReplace('\s*Is\sthis\sinteresting\?Interesting\?YesNo\|', Value, '', false);
      Value := RegExprSetReplace('Spoilers\sThe\strivia\sitems?\sbelow\smay\sgive\saway\simportant\splot\spoints\.\s*', Value, #13#10 + 'Spoilers:' + #13#10, false); 
      Value := RegExprSetReplace('\s*See\salsoGoofs.*', Value, '', false);      

      if (GetOption('Trivia') = 1) or (GetOption('Trivia') = 2) then
        Value := ' ' + Value;

      case GetOption('Trivia') of
        1,3:
          begin
            if GetField(fieldDescription) <> '' then
              Value := GetField(fieldDescription) + #13#10 + #13#10 + 'IMDB TRIVIA:' + Trim(Value)
            else
              Value :=  'IMDB TRIVIA: ' + Value;
            SetField(fieldDescription, Value);
          end;
        2,4:
          begin
            if GetField(fieldComments) <> '' then
              Value := GetField(fieldComments) + #13#10 + #13#10 + 'IMDB TRIVIA:' + Trim(Value)
            else
              Value :=  'IMDB TRIVIA: ' + Value;
            SetField(fieldComments, Value);
          end;
      end;
    end;
  end;
  // Awards
  if (GetOption('Awards') > 0) then
  begin
    ImportAwards();
  end;
  // Rating
  if CanSetField(fieldRating) then
  begin
//   (Remove the "//" of the next two lines beginning with "Value" and add "//" to the following two lines beginning with Value
//   if you would like to import arithmetic ratings instead of IMDB's user ratings)
//    Value := ConvertToASCII(GetPage(MovieURL + '/ratings'));
//    Value := TextBetween(Value, 'Arithmetic mean = ', '. ');
    Value := TextBetween(PageText, '<h5>User Rating:</h5>', '</b>');
    Value := TextBetween(Value, '<b>', '/');
    if Value <> '' then
      if (GetOption('RoundRating') = 1) then 
        SetField(fieldRating, IntToStr(Round(StrToFloat(Value))))
      else 
        SetField(fieldRating, Value);
  end;
  if GetOption('UserRatings') > 0 then
    begin
      Value := TextBetween(PageText, '<a href="ratings" class="tn15more">', '</a>');
      if Value <> '' then
        Value := 'User Rating: ' + GetField(fieldRating) + ' out of 10  (with ' + Value + ')';
      if (GetOption('UserRatings') = 1) and (Value <> '') and (CanSetField(fieldMediaType)) then
        SetField(fieldMediaType, Value);
      if (GetOption('UserRatings') = 2) and (Value <> '') then
        SetField(fieldComments, GetField(fieldComments) + #13#10 + #13#10 + Value);
    end;
  // Classification
  if CanSetField(fieldCertification) then
  begin
    if UserCountry = '' then
      ShowMessage('Country not set for classification selection - Click "Editor" tab in the scripting window, then select your country by modifying the required line as explained in the first lines of the code')
    else
    begin
      Value := TextBetween(PageText, '<a href="/search/title?certificates=', '</div>');
      Value := TextBetween(Value, UserCountry + ':', '</a>');      
      if Value <> '' then
        SetField(fieldCertification, Value);
    end;
  end;
  // MPAA rating
  if (GetOption('MPAA') > 0) then
  begin
     Value := FullTrim(TextBetween(PageText, '<h5><a href="/mpaa">MPAA</a>:</h5>', '</div>'));
     if Pos('<p>', Value) > 0 then
       Value := TextBetween(Value, '<p>', '</p>');
     HTMLRemoveTags(Value);
     Value := FullTrim(Value);
     // Value := TextAfter(Value, #13#10);
     if Value <> '' then
     begin
      if GetOption('MPAA') = 1 then
        SetField(fieldMediaType, TextBetween(Value, 'Rated ', ' '))
      else
        SetField(fieldComments, GetField(fieldComments) + #13#10 + #13#10 + Value);
     end;
  end;
end;

// ***** Imports lists like Genre, Country, etc. depending of the selected option *****

function ImportList(PageText: string; MultipleValues: Integer; StartTag: string): string;
var
  Value, Value2: string;
begin
  if MultipleValues = 0 then
  begin
    Value := TextBetween(PageText, StartTag, '</a>');
    Value2 := TextAfter(Value, '">');
  end
  else
  begin
    Value := TextBetween(PageText, StartTag, '</div>');
    Value2 := TextBefore(Value, '<a class="tn15more inline"', '');
    if Value2 = '' then
      Value2 := Value;
    Value2 := TextAfter(Value2, '">');
    HTMLRemoveTags(Value2);
    if MultipleValues = 1 then
      Value2 := StringReplace(Value2, ' | ', ', ');
    if MultipleValues = 2 then
      Value2 := StringReplace(Value2, ' | ', ' / ');
    if MultipleValues = 3 then
      Value2 := '';
  end;
  Value2 := StringReplace(Value2, #13#10, '');
  Value2 := StringReplace(Value2, ' , ', ', ');
  HTMLDecode(Value2);
  Result := FullTrim(Value2);
end;

// ***** functions to import the different pictures kinds, depending of the option selected by user *****

//Import small picture from IMDB movie main page
function ImportSmallPicture(PageText: string): Boolean;
var
  Value: string;
begin
  Result := False;
  Value := TextBetween(PageText, '<div class="photo">', '</div>');
  if (Value <> '') and (Pos('"Poster Not Submitted"', Value) = 0) then
  begin
    Value := TextBetween(Value, 'src="', '"');
    if Value <> '' then
    begin
      GetPicture(Value);
      Result := True;
    end;
  end;
end;

//Import large image from link on IMDB movie main page
function ImportLargePicture(PageText: string): Boolean;
var
  Value: string;
begin
  Result := False;
  Value := TextBetween(PageText, '<div class="photo">', '</div>');
  if (Value <> '') and (Pos('"Poster Not Submitted"', Value) = 0) and (Pos('<a name="poster" href="/rg/action-box-title/primary-photo', Value) > 0) then
  begin
    Value := TextBetween(Value, 'href="/rg/action-box-title/primary-photo', '"');
    if Value <> '' then
    begin
      Value := 'http://www.imdb.com' + Value;
      Value := GetPage(Value);
      Value := TextBetween(Value, 'primary-img', '</div>');
      if (Value <> '') and (Pos(' src="', Value) > 0) then
      begin
        Value := TextBetween(Value, ' src="', '"');
        GetPicture(Value);
        Result := True;
      end;
    end;
  end;
end;

//Image not available, import "No Poster Available" icon
function ImportPictureNotAvailable(PageText: string): Boolean;
var
  Value: string;
begin
  Result := False;
  Value := TextBetween(PageText, '<div class="photo">', '</div>');
  if (Value <> '') and (Pos('"Poster Not Submitted"', Value) > 0) then
  begin
    Value := TextBetween(Value, 'src="', '"');
    if Value <> '' then
    begin
      GetPicture(Value);
      Result := True;
    end;
  end;
end;

//Image from DVD Details Page
function ImportDvdDetailsPicture: Boolean;
var
  Value, PageText: string;
begin
  Result := False;
  PageText := ConvertToASCII(GetPage(MovieUrl+'/dvd'));
  Value := TextBefore(PageText, '.jpg alt="', '<table class="dvd_section" cellpadding="10"><tr>');
  if Value <> '' then
  begin
    Value := TextBetween(Value, '<a href="', '">');
    if Value <> '' then
      Result := ImportFromAmazonRedirect('http://www.imdb.com' + Value);
  end;
end;

//Image from Merchandising Links (/sales) Page
function ImportMerchandisingPicture: Boolean;
var
  Value, PageText: string;
begin
  Result := False;
  PageText := ConvertToASCII(GetPage(MovieUrl+'/sales'));
  Value := TextBefore(PageText, '.jpg" width=', '<td class=w_rowtable_colcover><a href="');
  if Value <> '' then
  begin
    Value := TextBefore(Value, '"><img', '');
    if Value <> '' then
      Result := ImportFromAmazonRedirect('http://www.imdb.com' + Value);
  end;
end;

function ImportFromAmazonRedirect(Url: string): Boolean;
var
  Value, PageText: string;
begin
  Result := False;
  PageText := ConvertToASCII(GetPage(Url));
  Value := TextBetween(PageText, '<td id="prodImageCell"', '" id="prodImage"');
//  Value := StringReplace(TextAfter(Value, ' src="'), '_AA240', '');
  Value := StringReplace(Value, TextBetween(Value, '_AA', '_'), '');
  Value := StringReplace(TextAfter(Value, ' src="'), '_AA', '');
  if Value <> '' then
  begin
    GetPicture(Value);
    Result := True;
  end;
end;

// ***** Gets summaries for the movie, based on the plot outline given in parameter (that contains the URL to more summaries) *****

function ImportSummary(PlotText: string): string;
var
  Address, Value, Value2, Value3, Value4, PageText, Longest: string;
begin
  Address := TextBetween(PlotText, '<a class="tn15more inline" href="', '" ');
  if (Address = '') or (GetOption('DescriptionSelection') = 0) or (GetOption('DescriptionSelection') = 3) then
  begin
    Value3 := FullTrim(TextBefore(PlotText, '<a class="tn15more inline"', ''));
    if Value3 = '' then
      Value3 := FullTrim(PlotText);
    HTMLRemoveTags(Value3);
    HTMLDecode(Value3);
  end;
  if (Address <> '') and (GetOption('DescriptionSelection') > 0) then
  begin
    PageText := ConvertToASCII(GetPage('http://www.imdb.com' + Address));
    PickListClear;
    Longest := '';
    //Value := TextBetween(PageText, '<p class="plotpar">', '</p>');
    Value := TextBetween(PageText, '<p class="plotSummary">', '</p>');
    PageText := RemainingText;
    while Value <> '' do
    begin
      //Value := TextBefore(Value, '<i>', '');
      Value := FullTrim(StringReplace(Value, #13#10, ' '));
      Value := StringReplace(Value, '<br>', #13#10);
      HTMLRemoveTags(Value);
      HTMLDecode(Value);
      while Pos('  ', Value) > 0 do
        Value := StringReplace(Value, '  ', ' ');
      if Length(Value) > Length(Longest) then
        Longest := Value;
      PickListAdd(FullTrim(Value));
      //Value := TextBetween(PageText, '<p class="plotpar">', '</p>');
      Value := TextBetween(PageText, '<p class="plotSummary">', '</p>');
      PageText := RemainingText;
    end;
    if (GetOption('BatchMode') > 0) or (GetOption('DescriptionSelection') = 2) then
      Value4 := Longest
    else
    begin
      if not PickListExec('Select a description for "' + GetField(fieldOriginalTitle) + '"', Value4) then
        Value4 := '';
    end;
  end;
  if Value3 <> '' then
  begin
     if Value4 <> '' then
     begin
        Result := Value3 + #13#10 + #13#10 + Value4;
     end
     else
     begin
        Result := Value3;
     end;
  end
  else
  begin
     Result := Value4;
  end;
end;

// Import awards
procedure ImportAwards;
var
  IndexPage: TStringList;
  PageText, FullValue, Block, Value, Row, Outcome, Details, AwardShow, PageText1: string;
  Year, Result, Award, Category: string;
begin
  sleep(50);
  Value := MovieUrl;
  PageText := ConvertToASCII(GetPage(Value+'/awards'));
  
  repeat

    AwardShow := TextBetween(PageText, '<h3>', '</h3>');

    if Pos('User Lists', AwardShow) > 0 then              
      break;
      HTMLRemoveTags(AwardShow);
      HTMLDecode(AwardShow);
      AwardShow := FullTrim(AwardShow);
      AwardShow := StringReplace(AwardShow, #13, '');
      AwardShow := StringReplace(AwardShow, #10, '');
      FullValue := FullValue + #32 + AwardShow + #13#10; 
      PageText1 := TextBetween(PageText, '<table class="awards"', '</table>');
      PageText := RemainingText;
      Block := PageText1;

      repeat
          
        Row := TextBetween(Block, '<tr>', '</tr>');
        Block := RemainingText;
 
        Outcome := TextBetween(Row, '<b>', '</b>');
        Category := TextBetween(Row, '<span class="award_category">', '</span>');
        Award := TextBetween(Row, '<td class="award_description">', '</td>');
        Details := TextBetween(Row, '<div class="award_detail_notes">', '</div>');
            
        If Outcome <> '' then
          FullValue := FullValue + #32 + Outcome + #13#10;
            
        If Category <> '' then
          FullValue := FullValue + #32 + Category;
            
        If Award <> '' then
          Award := StringReplace(Award, '<br />', #32);
          
          HTMLRemoveTags(Award);
          HTMLDecode(Award);
          Award := StringReplace(Award, #13, '');
          Award := StringReplace(Award, #10, '');                         
          Award := FullTrim(Award);        
          Award := StringReplace(Award, '  ', '');
          FullValue := FullValue + #32 + Award;
              
        If Details <> '' then
          HTMLRemoveTags(Details);
          HTMLDecode(Details);
          Award := StringReplace(Details, #13, '');
          Award := StringReplace(Details, #10, '');                         
          Award := FullTrim(Details);

          FullValue := FullValue + #32 + Details + #13#10;

     until (Row = '');  
  until (PageText1 = '');
  
 // ShowInformation(FullValue);
  
  if FullValue <> '' then
    case GetOption('Awards') of
      1:
        begin
          if GetField(fieldDescription) <> '' then
            Value := GetField(fieldDescription) + #13#10 + #13#10 + 'AWARDS: ' + #13#10 + FullValue
          else
            Value := 'AWARDS: ' + FullValue;
          SetField(fieldDescription, Value);
        end;
      2:
        begin
          if GetField(fieldComments) <> '' then
            Value := GetField(fieldComments) + #13#10 + #13#10 + 'AWARDS: ' + #13#10 + FullValue
          else
            Value := 'AWARDS: ' + FullValue;
          SetField(fieldComments, Value);
        end;
    end;
end;

// ***** beginning of the program *****

begin
  // Check StringUtils1 version and update if < version 7
  if StringUtils1_Version < 7 then
    begin
      if ShowWarning('Old version of file "Stringutils1.pas" detected: v'+IntToStr(StringUtils1_Version)+#13#10+'The script requires at least version 7.'+#13#10+'Download and install latest version now ?') = True then
        begin
          UpdateFile := TStringList.Create;
          UpdateFile.Text := GetPage('http://update.antp.be/amc/scripts/StringUtils1.pas');
          UpdateFile.SaveToFile(dirScripts + 'StringUtils1.pas');
          UpdateFile.Free;
          ShowInformation('StringUtils1 has been updated. Please restart IMDB script now. Thank you.');
          Exit;
        end
      else
        begin
          ShowInformation('You can download latest version of "StringUtils1.pas" using script "update scripts" or via http://update.antp.be/amc/scripts');
          Exit;
        end;
    end;
  // Check for current AMC version
  if CheckVersion(4,2,0) then
  begin
    MovieName := '';
    if GetOption('BatchMode') = 2 then
    begin
      MovieName := GetField(fieldURL);
      if Pos('imdb.com', MovieName) = 0 then
        MovieName := '';
    end;
    if MovieName = '' then
      MovieName := GetField(fieldOriginalTitle);
    if MovieName = '' then
      MovieName := GetField(fieldTranslatedTitle);
    if GetOption('BatchMode') = 0 then
    begin
      if not Input('IMDB Import', 'Enter the title or the IMDB URL of the movie:', MovieName) then
        Exit;
    end
    else
      Sleep(500);
    if MovieName <> '' then
    begin
      if Pos('imdb.com', MovieName) > 0 then
        AnalyzeResultsPage(MovieName)
      else if RegExprSetExec('tt[0-9]+', moviename) then
        AnalyzeResultsPage('http://www.imdb.com/title/' + MovieName)
      else
      begin
        MovieName := UTF8Encode(StringReplace(MovieName, '&', 'and'));
        if (GetOption('BatchMode') > 0) or (GetOption('PopularSearches') = 1) then
          AnalyzeResultsPage(PopularTitleSearchURL + UrlEncode(MovieName))
        else
          AnalyzeResultsPage(FullTitleSearchURL + UrlEncode(MovieName));
      end;
    end;
  end
  else
    ShowMessage('This script requires a newer version of Ant Movie Catalog (at least the version 4.2)');
end.

Posted: 2014-10-29 10:48:02
by antp
Thanks!

Posted: 2014-11-03 08:35:29
by lsstan
Hi elman,
I just installed your updated IMDB script (v3.84), and I want to thank you for the fixes you've done. I especially like that the 'interesting' strings are automatically removed.

Thanks!!
:grinking:

Posted: 2014-11-03 17:00:43
by elman
You are welcome :)

Posted: 2014-11-23 19:22:20
by elman
Just updated with one tiny fix. Text "Spoilers" in trivia was no removed if only one spoiler exists.

Posted: 2014-11-23 20:03:49
by Raoul_Volfoni
Hi elman,

Thanks for this update :)
If you have not seen this topic, it would be better to adjust some lines in the script.

In the "beginning of the program"
Change this line

Code: Select all

  if CheckVersion(4,1,0) then
to

Code: Select all

  if CheckVersion(4,2,0) then
and this line

Code: Select all

    ShowMessage('This script requires a newer version of Ant Movie Catalog (at least the version 4.1)'); 
to

Code: Select all

    ShowMessage('This script requires a newer version of Ant Movie Catalog (at least the version 4.2)');
Thanks in advance.

Posted: 2014-11-24 20:52:11
by elman
Ah, I thought that setting Requires=4.2.0 in Infos would be enough.

Anyway, I fixed it. Script is updated in post above and to be on the safe side, version is updated to 3.86 (I also fixed couple more bugs).