[ antp.be > Forum ]

[ FAQFAQ - SearchSearch - MemberlistMemberlist - UsergroupsUsergroups - ]

[ RegisterRegister - ProfileProfile - Log in to check your private messagesLog in to check your private messages - Log inLog in ]


[REQ] Batch Editing for Writer, Composer Fields?

 
Post new topic   Reply to topic    www.antp.be Forum Index -> Ant Movie Catalog > Comments
Author Message
Lemaireus
PostPosted: 2015-02-17 15:02:37    Post subject: [REQ] Batch Editing for Writer, Composer Fields? Reply with quote
View user's profile Send private message The writer and composer fields have become available in v4.2.0.2, but I had a collection running over 3000 titles in AMC before that version was released. To import or fill in the writer and composer individually for each title would be cumbersome and extremely time-consuming.

Is there a way to batch edit or import data for selected titles, say a hundred selected files in one go, for selected fields such as writer/composer? If yes, I'd love to have some information on how to do that. If such an option does not exist, then it may perhaps be worth considering as another feature to add for the next release.

Looking forward to some ideas on this.
antp
Site Admin
PostPosted: 2015-02-17 15:52:45    Post subject: Reply with quote
View user's profile Send private message Visit poster's website Batch-edit, yes, via script... but edit to which value?
Batch-import can be done if you have the IMDb (or another) URL stored in the URL field, but it would still take time, since the script would need to re-download the page for each movie to getting the info.
Lemaireus
PostPosted: 2015-02-17 19:08:26    Post subject: Reply with quote
View user's profile Send private message I want to add data to the "Writer" and "Composer" fields for about 2500 titles. If it is possible to add that data for 100 films in one step by choosing one set of options that'll do the work. Importing can take as long as necessary as long as one doesn't have to import separately for every single title.

I have the IMDB and other scripts (F6 option, have also tried Shift+F6 without success), but I haven't been able to run those scripts for more than one title at one time. And that means that in order to fill in writer/composer for 2500 titles I'll have to run the IMDB script 2500 times! It is possible to select multiple titles and run the scripts but the import has to be clicked separately for each title. The whole process takes so much time that adding 2 or 3 values via manual copy-paste would be quicker.

Is there a quicker way to do this for writer, composer, certification?
antp
Site Admin
PostPosted: 2015-02-18 12:09:32    Post subject: Reply with quote
View user's profile Send private message Visit poster's website To apply a script to more than one movie, select these movies before opening the script window.
In that window there is also an option ("movies to include", on the right) where you can specify that you want to apply to all movies rather than selection.

Then when the IMDb script is selected, on the right you have some options. One is "Batch mode". It allows to suppress confirmation dialogs and use the IMDb URL from the URL field instead of doing a title search. But as I said previously, to have this working fine the movies must have the IMDb URL stored there (I assume it is the case?) so the script can fetch the right movie info without having to ask you to confirm the title search.

Be sure to also uncheck all the fields that you don't want to change (also on the right of that window), keeping only writer/composer/certification checked.

Maybe do first the test by selecting a few movies, then if it works fine do it either on all or on larger sets.
Lemaireus
PostPosted: 2015-02-18 19:13:42    Post subject: Reply with quote
View user's profile Send private message Thank you, Antoine, for helping out. The steps you suggest are exactly the steps I have been taking in trying to get the job done, but I doubt if I would have been able to communicate them as concisely and clearly as you do.

The trouble I'm running into, as you anticipate, is that I do not have the IMDb URL in most of the titles: I use All Movie Guide as my default information import site, and their title search function has changed over time and the old URLs do not work anymore.

So it seems it is going to be the long way for me, at least for now, courtesy the URL issue. Thank you once again for your help.
antp
Site Admin
PostPosted: 2015-02-18 20:01:16    Post subject: Reply with quote
View user's profile Send private message Visit poster's website Do the AllMovieGuide script still work and does it provide info for these new fields?

If yes, maybe we could easily convert the old URLs to the new ones with a script (just post an example of the old-style URL and the corresponding new one) and do the update using AllMovieGuide.

If no, then you can run the IMDb script without activating the batch mode: you'll have to select the movie in the search result each time, but it is still rather automatic as process, as the script pass to the next movie each time by itself, and pre-fills the title.
Lemaireus
PostPosted: 2015-02-18 20:33:13    Post subject: Reply with quote
View user's profile Send private message Thanks, Antoine, it would be terrific if this could be worked into the All Movie Guide script. I'm using the updated version of AMG script, updated twice by Raoul and it gets the information for all the new fields after RV's update.

The new AMG URLs all work on the same lines, but the older ones have changed several times and work along three different patterns, here are examples for three titles which illustrate the three older version URLs along with the updated new ones:

Title 1: The Angels' Share:

Old URL: http://www.allrovi.com/movies/movie/the-angels-share-v561662
New URL: http://www.allmovie.com/movie/the-angels-share-v561662

Title 2: Angels and Demons:

Old URL: http://www.allmovie.com/work/angels-amp-demons-381322
New URL: http://www.allmovie.com/movie/angels-demons-v381322

Title: 3: Anger Management:

Old URL: http://allmovie.com/cg/avg.dll?p=avg&sql=1:276390
New URL: http://www.allmovie.com/movie/anger-management-v276390

If the writer, composer, certification fields for 2500 to 3000 titles could be imported from AMG in batches of 50 to 100 titles in one go via a script then that would be a terrific solution for the problem I'm running into. Such a script would reduce the work from two hours a day for two months to perhaps four hours flat!

Keeping my fingers crossed for some good news.
antp
Site Admin
PostPosted: 2015-02-19 11:57:51    Post subject: Reply with quote
View user's profile Send private message Visit poster's website The two old-style "allmovie.com" URLs seem to work, so I guess they can be used as they are.
The "allrovi.com" URL however does not work.
Here is a script to apply to all the movies to change these URLs:

Code:
program NewScript;
var
  url: string;
begin
  url := GetField(fieldUrl);
  url := StringReplace(url, 'allrovi.com/movies', 'allmovie.com');
  SetField(fieldUrl, url);
end.


I updated the AllMovieGuide to version 0.7.4 to add a "BatchMode" option that you can set to 1 so it will use the URL if available, or else it will prompt for the title like the normal working mode.
Lemaireus
PostPosted: 2015-02-19 15:32:12    Post subject: Reply with quote
View user's profile Send private message Thank you, Antoine, for the URL updating script: that works perfectly. I replaced all the old "allrovi.com" URLs with the script you have provided.

The updated All Movie Guide script v0.7.4, however, does not seem to be working as desired. If I import data for a NEW title, then the new script works perfectly, importing all the relevant/selected fields, including writer/composer/certification.

But if I am trying to UPDATE an OLD title by importing data only for writer/composer/certification then the new AMG script imports neither the writer nor the composer. The certification information does get imported successfully. I cannot figure out the reason for these irregular results.

Perhaps if you tried running the AMG script in these two ways--firstly for a new title, in which case everything works perfectly; the second time only for updating writer/composer/certification fields in an old title, in which case it doesn't work for writer/composer fields--you'll be able to figure out where the problem may be.

PS 1: I should have mentioned that I'm running the AMG script in batch mode.
PS 2: Tried running the script in non-batch mode after posting PS 1 above, but the result doesn't change: writer/composer fields do not get updated.
antp
Site Admin
PostPosted: 2015-02-19 15:57:12    Post subject: Reply with quote
View user's profile Send private message Visit poster's website Indeed, the way the script is made it must be allowed to modify either comment or description fields.
This is because in previous versions of AMC, these fields are used to store writer etc.
So if you can update one of these two without losing anything, check one of these two.
Or else I can fix that "bug" in the script.

edit: I see other flaws similar than that in the script, I'll fix these at the same time.
antp
Site Admin
PostPosted: 2015-02-19 16:07:28    Post subject: Reply with quote
View user's profile Send private message Visit poster's website I updated the script to 0.7.5 : it should now work without having to check comment or description fields.
I fixed some flaws: some pages were downloaded twice, or downloaded even if not needed.
I hope I didn't break anything, check that it still work in the different cases wink
Lemaireus
PostPosted: 2015-02-19 18:22:13    Post subject: Reply with quote
View user's profile Send private message Terrific piece of work! I've tested the script with new titles, and have also used the batch mode to update writer/composer/certification for old ones via AMG, and thus far everything seems to be working perfectly grin If I notice any issue at any point in the coming days I'll post it here, but for now it is working perfectly as it should.

Thank you once again, Antoine, you've saved me hours and hours of work grin kind
Lemaireus
PostPosted: 2015-02-20 13:14:41    Post subject: Reply with quote
View user's profile Send private message The problem with the AMG script persists: it picks up writer and composer only sporadically. I'm doing 10 titles on one go, and it picks up writer+composer data only about 2 or 3 times out of 10. The certification data comes through perfectly fine. I guess the script is still somewhat buggy, but I can't see why it should import the data occasionally while failing to do that most of the time.

Importing data for new titles still works perfectly well, the problem occurs only when updating writer+composer for old titles.
antp
Site Admin
PostPosted: 2015-02-20 17:37:07    Post subject: Reply with quote
View user's profile Send private message Visit poster's website And those for which it fails, does it fails each time? Or only sometimes?
If each time, their URL must have a problem then.
If only some time, maybe the site does not like when too many requests are done, for that we can add a pause in the script between each movie.
Lemaireus
PostPosted: 2015-02-20 19:56:23    Post subject: Reply with quote
View user's profile Send private message Yes, Antoine, URL seems to be the primary issue: my catalog has AMG URLs in three formats and I ran tests using all the three formats with the following results:

TEST 1: Title: Brake. Latest AMG URL Format: http://www.allmovie.com/movie/brake-v556315

The script was run in batch mode.

All the data was filled up already, except three empty fields: writer, composer, certification. When I updated with only writer, composer, and certification selected--'allow to clear fields' also checked in 'execution options'--all three filed imported the correct data perfectly.

Checked test 1 result with another title and got exactly the same result: all the three fields imported the data perfectly.

TEST 2: Title: Bottle Rocket. Old Format 1 AMG URL: http://www.allmovie.com/work/bottle-rocket-135848

Script run in batch mode. Attempt to update selected writer, composer, and certification fields, which were the only empty ones. The imported data filled up all the three fields perfectly.

Ran 'test 2' again with two more titles and the data for all three fields--writer, composer, certification--got imported without any error.

TEST 3: Title: Bound. Old Format 2 AMG URL: http://www.allmovie.com/cg/avg.dll?p=avg&sql=1:135539

Latest AMG URL for 'Bound': http://www.allmovie.com/movie/bound-v135539/

Script run in batch mode. Attempted to update empty writer, composer, certification fields. Writer and composer remained empty, certification got filled up properly.

Ran the script once again, with the URL change option selected, no change in results, same as above. The URL did not change.

Unchecked the URL updating option and ran the script a third time. Again no change in results, the same as above: writer and composer empty, certification filled up.

Checked the result for 'test 3' with two more titles which had identical old format 2 AMG URLs and the results were exactly the same as mentioned above in the test 3 title 'Bound'.

Results seem to be self-explanatory: TEST 3 'Old Format 2 AMG URL' needs to be converted to the latest format AMG URL; perhaps you could add a line to that effect in the URL editing script you posted earlier and it should be smooth sailing thereafter. In fact, if that script is getting updated, then why not add an option for updating both the old formats to the latest one? That way the script will standardize all the URLs in the latest format through the entire catalog.
Lemaireus
PostPosted: 2015-02-21 15:26:27    Post subject: Reply with quote
View user's profile Send private message Antoine, looking at your script I thought I'll try to do a copy-paste job for 'Old Format 2 AMG URL' and, to my complete surprise, it worked! Here is the script for updating the URL format of TEST 3 example:

Code:
program NewScript;
 var
   url: string;
 begin
   url := GetField(fieldUrl);
   url := StringReplace(url, 'allmovie.com/cg/avg.dll?p=avg&sql=1:', 'allmovie.com/movie/v');
   SetField(fieldUrl, url);
 end.


I've tested the script with three titles and it imports the available data without any errors.

However, one problem remains. There is another series of URLs that I hadn't noticed before. Here are three examples:

Old Format 3 AMG URL: http://allmovie.com/cg/avg.dll?p=avg&type=2&srch=Passion+of+the+Christ

Updated URL: http://www.allmovie.com/movie/the-passion-of-the-christ-v290960

Old Format 3 AMG URL: http://allmovie.com/cg/avg.dll?p=avg&type=2&srch=Wages%20Fear

Updated URL: http://www.allmovie.com/movie/the-wages-of-fear-v53139

Old Format 3 AMG URL: http://allmovie.com/cg/avg.dll?p=avg&type=2&srch=You+Can+Count+on+Me

Updated URL: http://www.allmovie.com/movie/you-can-count-on-me-v187097

My rudimentary copying job did not work with this set of URLs. If preparing a script to update these URLs to the latest format is not too much of a problem then any help on this front would be much appreciated.
antp
Site Admin
PostPosted: 2015-02-23 15:02:16    Post subject: Reply with quote
View user's profile Send private message Visit poster's website That kind of URL can't actually be converted easily: they are just URL for searching and not direct movie URLs. Notice that there is no movie number (e.g. 290960 for the first one). I suppose that when the search was returning only one result, the script saved that URL directly.
Lemaireus
PostPosted: 2015-02-24 17:04:23    Post subject: Reply with quote
View user's profile Send private message Yes, I also noticed the missing number, and I suppose that was the primary reason why my copying your script didn't work in the case of this particular URL.

The variety of AMG URLs are giving more trouble than can be dealt with easily so I'm going with your earlier suggestion of importing the selected data from IMDb. Cannot work in batch mode because of URL discrepancies, but the script works very, very well and pretty much automatically except for clicking each title. Manage to do about 100 titles in 60 minutes or so, should be done in a few days.

Thanks again, Antoine.
Display posts from previous:   
View previous topic :: View next topic  
Post new topic   Reply to topic    www.antp.be Forum Index -> Ant Movie Catalog > Comments All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2006 phpBB Group
Template made by antp