Plugin Cafe Homepage
Forum Home Forum Home > Plugin Cafe > PYTHON Development
  New Posts New Posts
  FAQ FAQ  Forum Search

[SOLVED]Know the change of bitmap

Author
Message
  Topic Search Topic Search
gr4ph0s View Drop Down
Member
Member


Joined: 2015 Jul 07
Location: France
Online Status: Offline
Posts: 437
Direct Link To This Post Topic: [SOLVED]Know the change of bitmap
    Posted: 2016 Dec 15 at 1:01am
Hello I would like to know if it's possible to catch the msg when a bitmap is reloaded / created / deleted

Here is my project. We currently have huge slow down when we use the texture manager from c4d(cause it's refresh every time you move it or you do something in the UI), and since we got a lot of relative texture path wich can be on a lot of directory.
It often just take 40/50s just to show up and as said at each move we have to rewait 40/50s wich is not possible for us.

So I firstly thinked about rebuilding the texture manager witout the auto refresh... (Don't know if it's the good place for c4d feature request but it should be really good to have an option for disallow this.)
Then I thinked about another way for removing extra time. Wich will be at each time user modifiate path / reload a Xbitmap it will search for the path and then add to an array.

Technically I thinked about using messageplugin to catch the event about the change on any xbitmap (but is there a way for knowing which BaseList2d invok the event?) and then create background thread for the search like that it will not freeze the ui.


Again I'm not sticky to python... But since I never really dealing with thread (even if I understand the concept I never have to handle them. So handle them in c++ afraid me a bit ^^)

Btw can you confirm me the order for checking texture are still this one
Originally posted by Andreas Block

suggestedfolder + srcname

Thanks in advance ! :)
Gonna look to the chrismas competition for a training about threading gonna be a good idea :)


Edited by gr4ph0s - 2017 Jan 17 at 3:37am
Back to Top
gr4ph0s View Drop Down
Member
Member


Joined: 2015 Jul 07
Location: France
Online Status: Offline
Posts: 437
Direct Link To This Post Posted: 2016 Dec 16 at 3:06am
Anyway a litlle feature request it would be nice to have ExistPreset function avialbe in python. have https://developers.maxon.net/docs/Cinema4DCPPSDK/html/class_s_d_k_browser.html#a090f35abf2a3d9a6d6b73faa1e96001f

As a workaround I'm doing this but I think is way more slow than just a check.
        src = c4d.bitmaps.BaseBitmap()
        if src.InitWith(preset)[0]==c4d.IMAGERESULT_OK: 


Edited by gr4ph0s - 2016 Dec 16 at 3:10am
Back to Top
Yannick Puech View Drop Down
Forum Moderator
Forum Moderator


Joined: 2011 Apr 13
Location: Spain
Online Status: Offline
Posts: 1143
Direct Link To This Post Posted: 2016 Dec 16 at 4:03am
Hi,

There's no specific message when a bitmap shader is modified. You have to catch EVMSG_CHANGE and check each bitmap shader in the document.
To optimize the search for updated shaders, check shader.GetDirty(c4d.DIRTYFLAGS_DATA). The number returned is incremented every time the shader is changed.
And I think you shouldn't use multithreading in your first implementation.

You no longer have to manually build textures file paths. Since R16.50 c4d.GenerateTexturePath() is available.

About feature requests, you can make suggestions using this form.

I'll add ExistPreset() to our Python Parity list.



MAXON
Developer Support
Back to Top
gr4ph0s View Drop Down
Member
Member


Joined: 2015 Jul 07
Location: France
Online Status: Offline
Posts: 437
Direct Link To This Post Posted: 2016 Dec 16 at 4:43am
Thanks for the answerd.

For the moment I just rebuild the texture manager.

For my use c4d.GenerateTexturePath is not usuable. Since it's only a os.path.join like function or maybe I missunderstand it.
Just create a an standar material with a bitmap locate in a globalTexturePath
    docpath = doc.GetDocumentPath()
    srcname = doc.GetFirstMaterial().GetFirstShader()[c4d.BITMAPSHADER_FILENAME]
    suggestedfolder = None
    print c4d.GenerateTexturePath(docpath, srcname, suggestedfolder) #return None

But this is not my problem at the moment.
Moreover I don't want to use the GetAllAssets since it's very slow (took 1m15sec on my scene) and with my script event if I laod bitmap it took only 20sec. And adding like 1000 random texture just take 35sec while the GetAllAssets freeze during 3min...

I have another question. 
Is it possible to know if a shader is used by a material
At the moment I just grab all bitmapshader from all material.
But I have a shader inserted into a material wich is not used (I can't see it in any link of my shader).
Is there a way for identify shuch shader like that I can delete them from my list to check.

Thanks in advance ! :)

 



Edited by gr4ph0s - 2016 Dec 16 at 5:41am
Back to Top
Yannick Puech View Drop Down
Forum Moderator
Forum Moderator


Joined: 2011 Apr 13
Location: Spain
Online Status: Offline
Posts: 1143
Direct Link To This Post Posted: 2016 Dec 19 at 9:09am
Hi,

When you call GenerateTexturePath() pass an empty string for suggestedfolder instead of None. This way the the expected texture filename will be returned.
And GenerateTexturePath() doesn't just concatenate path and file names. It's really more complex then that and performs the steps you quoted in your initial post.

Materials, not shaders, are assigned to objects. So you can only check if a material is used, then you can discard it and all its shaders.
To check if a material is used, retrieve the state of its BIT_MATMARK like this:
import c4d

# First, reset BIT_MATMARK of all materials
mat = doc.GetFirstMaterial()
while mat: 
    mat.DelBit(c4d.BIT_MATMARK)
    mat = mat.GetNext()

# Send MSG_MULTI_MARKMATERIALS message to document
if doc.MultiMessage(c4d.MULTIMSG_ROUTE_BROADCAST, c4d.MSG_MULTI_MARKMATERIALS):
    mat = doc.GetFirstMaterial()
    while mat: 
        # Check BIT_MATMARK
        print "Material '" + mat.GetName() + "' Used: ", mat.GetBit(c4d.BIT_MATMARK)
        mat = mat.GetNext()



MAXON
Developer Support
Back to Top
gr4ph0s View Drop Down
Member
Member


Joined: 2015 Jul 07
Location: France
Online Status: Offline
Posts: 437
Direct Link To This Post Posted: 2016 Dec 19 at 9:42am
Thanks about GenerateTexturePath() gonna try on huge scene to check if it's slower than my implementation or not. (Since I'm not at work I will answerd to this only in few week but anyway thanks for the tips).
And it's seem to check if the file exist?Is it trustable or it's better to do a os.path.exists after it?
Does it also, check if presset exist? This could be a really better workaround for me to check presset file with this function than with the one I did before.


About the second point I really speak about shader not material.
It's like someone have done this.
mat = doc.GetFirstMaterial()
shd = c4d.BaseList2D(c4d.Xbitmap)
mat.InsertShader(shd)
c4d.EventAdd()
So that lead to a shader who is inserted into a material but wich is not used at anytime...

But your script would be usefull for another project then thanks you ! :)


Edited by gr4ph0s - 2016 Dec 19 at 9:51am
Back to Top
Yannick Puech View Drop Down
Forum Moderator
Forum Moderator


Joined: 2011 Apr 13
Location: Spain
Online Status: Offline
Posts: 1143
Direct Link To This Post Posted: 2016 Dec 20 at 3:55am
Originally posted by gr4ph0s

And it's seem to check if the file exist?Is it trustable or it's better to do a os.path.exists after it?
Yes GenerateTexturePath() checks if the texture file exists.

Originally posted by gr4ph0s

Does it also, check if presset exist? This could be a really better workaround for me to check presset file with this function than with the one I did before.
That's not what GenerateTexturePath() is meant to do. It returns the full path to a texture or another asset used in a document.

Originally posted by gr4ph0s

About the second point I really speak about shader not material.
It's like someone have done this.
mat = doc.GetFirstMaterial()
shd = c4d.BaseList2D(c4d.Xbitmap)
mat.InsertShader(shd)
c4d.EventAdd()
So that lead to a shader who is inserted into a material but wich is not used at anytime...
Thanks for the explanation. Checking if a shader inserted into a material is really used is a complex task.
The easiest checks would be to test if the shader is used in any of the material's channel.





Edited by Yannick Puech - 2016 Dec 20 at 3:57am
MAXON
Developer Support
Back to Top
gr4ph0s View Drop Down
Member
Member


Joined: 2015 Jul 07
Location: France
Online Status: Offline
Posts: 437
Direct Link To This Post Posted: 2017 Jan 04 at 3:28am
Thanks for the answer.
Finally I managed to get it working pretty fast and the most important not so much related to textures count using a combination of methods.
I list all textures and try to find them in all searchPath(see first post for know wich one).Like that I do only 1 os.walk(wich is the slower part of the code of course) and then for extra_checking I do GenerateTexturePath for some missings textures.
This finally give me the same output as the Texture manager from c4d.
But mine load in 18sec, where it took 1min30 with the default texture manager.

Moreover is there a way for use GenerateTexturePath not on the main thread? Because for the moment even if I use my own thread it's freeze everything :/
Here is some code maybe it would be more understandable, ofc this not my full code, even if I plan for a final open source project I would release it when it will be full working.

#here my threading attemps
class MyThread(c4d.threading.C4DThread):
    utility = Utility()

    texture_found = list()
    texture_to_relocate = list()
    texture_not_found = list()

    create_ui = None

    def Main(self):
        c4d.StatusSetSpin()
        self.texture_found, self.texture_to_relocate, self.texture_not_found = self.utility.get_texture_data(self.Get())
        c4d.StatusClear()

#the main function only a little part
class Utility(object):
    def get_texture_data(self, thread):
        all_tex = self.__get_all_texture()

        texture_found = []
        texture_to_relocate = []
        texture_not_found = []

        doc = c4d.documents.GetActiveDocument()
        docpath = doc.GetDocumentPath()
        suggestedfolder = str()
        for tex in all_tex:
            texture = c4d.GenerateTexturePath(docpath, tex["path"], suggestedfolder, bt=thread)

#And where I make the lunch of my Dialog
class Dialog(c4d.gui.GeDialog):
    utility = Utility()

    texture_found = list()
    texture_to_relocate = list()
    texture_not_found = list()

    can_be_closed = True
    thread_get_texture = None

    def get_all_textures(self, is_update):
        self.can_be_closed = False
        c4d.StatusSetSpin()

        self.thread_get_texture = MyThread()
        if is_update:
            self.thread_get_texture.create_ui = False
        else:
            self.thread_get_texture.create_ui = True

        self.thread_get_texture.Start()

    def CreateLayout(self):
        ###SOME STUFF###
        self.get_all_textures(False)
        return True

Thanks in advance ! :)


Edited by gr4ph0s - 2017 Jan 04 at 3:50am
Back to Top
Yannick Puech View Drop Down
Forum Moderator
Forum Moderator


Joined: 2011 Apr 13
Location: Spain
Online Status: Offline
Posts: 1143
Direct Link To This Post Posted: 2017 Jan 05 at 4:45am
With a similar code I can't get a freeze calling c4d.GenerateTexturePath() on a custom thread.
So I don't think the issue is inside the code you posted.
How do you check the custom thread has finished its processing?



MAXON
Developer Support
Back to Top
gr4ph0s View Drop Down
Member
Member


Joined: 2015 Jul 07
Location: France
Online Status: Offline
Posts: 437
Direct Link To This Post Posted: 2017 Jan 05 at 6:17am
I simply not check.
When the code is over at the end of my function I do A specialEventAdd()
Then in my dialog I catch this event and fill my UI.

I uplaoded on github a bigger part of my code. It's not 100% bug free, but I hope this will help you to understand a litlle bit better how everything works.
https://github.com/gr4ph0s/C4D-TextureChecker

Anyway thanks in advance ! :)

Back to Top
gr4ph0s View Drop Down
Member
Member


Joined: 2015 Jul 07
Location: France
Online Status: Offline
Posts: 437
Direct Link To This Post Posted: 2017 Jan 10 at 6:06am
Ok don't know exactly what change made this happen(you know this feling when something work when normally it should not ^^) but now my code is really executed into another thread so my UI is not anymore freezing.
I still get some crash regarding threading but it's 100% an error from me, I will came back if I don't find the solution.

I got another question. In our library we got some corrompteds textures. Wich load to an incapability of C4D to export/use them. Is there an easy way for find those textures excepting with a basebitmap->InitWith? (wich is pretty slow on big file).

Is there a way for listing all assets in a library (tipicaly do a os.walk on preset?) like that I could remove the corrompted data from the library.

Thanks in advance :)

EDIT:
I have figure out which line make c4d crash.
bmp = c4d.bitmaps.BaseBitmap()
def __test_valid_picture(self, picture_path):
ascii_path = picture_path.encode('utf8')
result = self.bmp.InitWith(ascii_path) #crash here
self.bmp.FlushAll()
if result[0] == c4d.IMAGERESULT_OK:
return True
else:
return False

The weirdest thing is I checked more than 350 textures. And c4d crash always with the same one. So maybe is not a crash related to the thread but to the texture (wich is 100% ok, render fine when appliqued to a plan).
And it's not crashing everytime. Sometime the problematic texture load correctly and not crash c4d.

I also updated the github if you want to check ;)
I still have to do some code cleanup.

EDIT 2:
Executing the code posted previously from the main thread don't seem to crash.

EDIT 3:
Having only the problematic texture in the scene and with my multi-threaded code it's seem to not crash
The most strange thing it's crashing while he already loaded the bitmap. And it seem to always crash on the 4th iteration of the map.
Look at my screen http://img11.hostingpics.net/pics/432714crash.png and I get this code
    def __test_valid_picture(self, picture_path):
        bmp = c4d.bitmaps.BaseBitmap()
        print 'b'
        ascii_path = picture_path.encode('utf8')
        result = bmp.InitWith(ascii_path)
        print 'a'
        bmp.FlushAll()
        del(bmp)
        if result[0] == c4d.IMAGERESULT_OK:
            return True
        else:
            return False
Is their some threading limitation for using InitWith in a custom thread?



Edited by gr4ph0s - 2017 Jan 12 at 2:14am
Back to Top
gr4ph0s View Drop Down
Member
Member


Joined: 2015 Jul 07
Location: France
Online Status: Offline
Posts: 437
Direct Link To This Post Posted: 2017 Jan 13 at 6:56am
Sorry for the triple post.
But GenerateTexturePath on another thread when image don't exist make c4d crash.
Here is a short exemple that make c4d crash
# encoding: utf-8
import c4d
import os
from ctypes import pythonapi, c_void_p, py_object

PLUGIN_ID = 1038562

class MyThread(c4d.threading.C4DThread):
    def __test_valid_picture(self, picture_path):
        bmp = c4d.bitmaps.BaseBitmap()
        ascii_path = picture_path.encode('utf8')
        result = bmp.InitWith(ascii_path)
        bmp.FlushAll()
        del(bmp)
        if result[0] == c4d.IMAGERESULT_OK:
            return True
        else:
            return False

    def get_texture_data(self, thread):
        doc = c4d.documents.GetActiveDocument()
        docpath = doc.GetDocumentPath()
        suggestedfolder = str()

        path = "qsdsqdsdqdqzz.jpg" #random name. in the case where picture don't exist.
        texture = c4d.GenerateTexturePath(docpath, path, suggestedfolder, bt=thread) #this line crash

        self.__test_valid_picture(path)

        return True

    def Main(self):
        c4d.StatusSetSpin()
        self.get_texture_data(self.Get())
        c4d.StatusClear()

        c4d.SpecialEventAdd(PLUGIN_ID, p1=1)


class Dialog(c4d.gui.GeDialog):
    thread_get_texture = None

    def CreateLayout(self):
        c4d.StatusSetSpin()

        if self.thread_get_texture is None:
            self.thread_get_texture = MyThread()
            self.thread_get_texture.Start(c4d.THREADMODE_ASYNC)
        else:
            self.thread_get_texture.End(True)
            self.thread_get_texture = MyThread()
            self.thread_get_texture.Start(c4d.THREADMODE_ASYNC)

        return True

    def AskClose(self):
        self.thread_get_texture.need_to_close = True
        self.thread_get_texture.End(True)
        return False

    def CoreMessage(self, id, msg):
        if id == PLUGIN_ID:
            P1MSG_UN = msg.GetVoid(c4d.BFM_CORE_PAR1)

            pythonapi.PyCObject_AsVoidPtr.restype = c_void_p
            pythonapi.PyCObject_AsVoidPtr.argtypes = [py_object]
            P1MSG_EN = pythonapi.PyCObject_AsVoidPtr(P1MSG_UN)

            if P1MSG_EN == 1:
                if self.thread_get_texture.IsRunning():
                    self.thread_get_texture.need_to_close = True
                    self.thread_get_texture.End()

                self.can_be_closed = True
        return True


class Launcher(c4d.plugins.CommandData):
    dialog = None

    def Execute(self, doc):
        if self.dialog is None:
            self.dialog = Dialog()
        return self.dialog.Open(dlgtype=c4d.DLG_TYPE_ASYNC, pluginid=PLUGIN_ID, defaultw=200, defaulth=50, xpos=-1,
                                ypos=-1)

    def RestoreLayout(self, sec_ref):
        if self.dialog is None:
            self.dialog = Dialog()
        return self.dialog.Restore(pluginid=PLUGIN_ID, secret=sec_ref)


if __name__ == "__main__":
    c4d.plugins.RegisterCommandPlugin(PLUGIN_ID, "00 - Texture Manager", 0, None, None, Launcher())

Thansk in advance ! :)


Edited by gr4ph0s - 2017 Jan 13 at 6:57am
Back to Top
Yannick Puech View Drop Down
Forum Moderator
Forum Moderator


Joined: 2011 Apr 13
Location: Spain
Online Status: Offline
Posts: 1143
Direct Link To This Post Posted: 2017 Jan 16 at 2:31am
Thanks for posting the code.
I'm afraid there's an internal issue passing bt thread parameter of c4d.GenerateTexturePath().
This bug will be fixed.



MAXON
Developer Support
Back to Top
gr4ph0s View Drop Down
Member
Member


Joined: 2015 Jul 07
Location: France
Online Status: Offline
Posts: 437
Direct Link To This Post Posted: 2017 Jan 16 at 3:29am
Using the python threading module can be a workaround? And is it really bad to use python threading module instead of the c4d one?
Moreover this fix will be aviable in all c4d versions(all version contening this functions ofc) or just R18?

Anyway thanks for your answer :)


Edited by gr4ph0s - 2017 Jan 16 at 3:30am
Back to Top
Yannick Puech View Drop Down
Forum Moderator
Forum Moderator


Joined: 2011 Apr 13
Location: Spain
Online Status: Offline
Posts: 1143
Direct Link To This Post Posted: 2017 Jan 17 at 3:26am
Using the python threading module can't be a workaround because GenerateTexturePath() only accepts a BaseThread.
It's recommended to use the c4d.threading classes from the Cinema Python API because these are the supported internally.

Note we don't backport fixes to past releases of Cinema 4D. So GenerateTexturePath() will be fixed in the next update of R18 and future releases.



MAXON
Developer Support
Back to Top
gr4ph0s View Drop Down
Member
Member


Joined: 2015 Jul 07
Location: France
Online Status: Offline
Posts: 437
Direct Link To This Post Posted: 2017 Jan 17 at 3:36am
Thanks for the infos !
Back to Top

Forum Jump Forum Permissions View Drop Down

Bulletin Board Software by Web Wiz Forums® version 9.61 [Free Express Edition]
Copyright ©2001-2009 Web Wiz

This page was generated in 0.141 seconds.