/TAGHI T ARANI/Supervisory Patent Examiner, Art Unit 2438 Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
1. The following is a non-Final Office Action in response to applicant’s arguments/filing filed on October 10, 2024
Claims 1-20 are pendingExaminer’s Note: The specification, in paragraphs 0055 and 0056, identifies the term, “image capturing module” as potentially being in the form of a camera that is physically mounted on a device. Therefore, an “image capturing device” is interpreted as hardware.
Foreign Priority
Acknowledgment is made of applicant's claim for foreign priority under 35 U.S.C. 119(a)-(d). The certified copy has been filed in the instant Application, filed on 10/10/2024.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/10/2024 was filed prior to the mailing date of the first office action. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Drawings
Acknowledgment is made of applicant’s drawings submitted on 10/10/2024.
Oath/Declaration
Acknowledgment is made of applicant’s oath submitted on 10/10/2024
Application Data Sheet
Acknowledgment is made of applicant’s application data sheet submitted on 10/10/2024.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
1.) Claims 1- 20 are rejected under 35 U.S.C 103 as being unpatentable over IDS supplied references, US 20210035342, Glaser in view of US 11282174, DeMaio
In regards to claim 1, Glaser teaches a device for processing an image, the device comprising:an image capturing module configured to obtain a first image capturing an external environment(US 20210035342, Glaser, para. 0079, wherein a method for automatic privacy preservation using CV-based interpretation of the environment can include collecting image data from an environment S310);a memory configured to store data(US 20210035342, Glaser, para. 0192, The communication channel 1001 interfaces with the processors 1002A-1002N, the memory (e.g., a random access memory (RAM)) 1003, a read only memory (ROM) 1004, a processor-readable storage medium 1005,);a communication interface arranged in data communication with a remote data storage(US 20210035342, Glaser, para. 0195, A network device 1008 may provide one or more wired or wireless interfaces for exchanging data and commands between the system and/or other devices, such as devices of external systems.); and a processor communicatively couplable with the image capturing module, the memory, and the communication interface(US 20210035342, Glaser, para. 0190, a system comprising of one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause a computing platform to perform operations comprising those of the system or method described herein such as: collecting image data; processing image data and thereby generating sanitized image data, which includes selecting a sanitization region in the image data and applying image sanitization to the sanitization region; and managing storage of image data.), and configured to:detect one or more regions of the first image, wherein the one or more regions include one or more objects of a predetermined type, and generate a second image corresponding to the first image, by blurring one or more regions of the second image each corresponding to the one or more regions of the first image(US 20210035342, Glaser, para. 0030 and 0079: [0030]- A primary example of image sanitization may include image redaction or blurring.[0079]- the method may be adapted to use a form of conditional image sanitization, wherein a method for automatic privacy preservation using CV-based interpretation of the environment can include collecting image data from an environment S310; detecting an object and or event registered for image sanitization S320; selecting the sanitization image region based on location of the object and/or event in the image data S332; applying image sanitization to the sanitization image region thereby generating sanitized image data S334 as shown in FIG. 16. ), wherein the processor is further configured to store the first image in the memory for a predetermined time(US 20210035342, Glaser, para. 0171, original image data (pre-sanitization) can be stored in a data storage system and then processing of the image data and generating sanitized image data is applied using the stored image data. In this manner, the original version of the image data can be preserved. The sanitized image data may be separately stored, communicated to another system, and/or used to replace the stored image data. In one variation, the sanitized image data may be stored separately for use with particular systems, while the original may be used as a canonical record. In another variation, image data meeting certain conditions may be replaced with sanitized image data, which functions to store partially sanitized and unsanitized image data. For example, image data may be updated to sanitized image data after some set time duration (e.g., 24 hours, 1 week, etc.)), and Glaser does not teach transmit the second image to the remote data storage via the communication interface However, DeMaio teaches transmit the second image to the remote data storage via the communication interface(US 11282174, DeMaio, col. 48, lines 34-44, The rewards-based, universal, integrated code base 203 can be configured to interact with a plurality of authorized user devices 208 and to perform operations including receiving, from a first user device operated by a first user, an image of a second user having a second user device, wherein the second user has a user profile with a privacy setting indicating that pictures or video taken of the second user by the first device should be blurred, based on the user profile, blurring the image of the second user taken by the first user device and transmitting a request to the second user device to unblur the image.). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of Glaser with the teaching of DeMaio because a user would have been motivated to enable sharing of personal information, taught by DeMaio, by using profile data to control image accessibility in the system taught by Glaser(DeMaio, col. 11, lines 31-49) In regards to claim 2, the combination of Glaser and DeMaio teach the device according to claim 1, wherein the processor is configured to store the first image in the memory in an encrypted format(US 20210035342, Glaser, para. 0030 and 0074: [0030]- Sanitization can involve fully or partially redacting, reducing resolution/fidelity, modifying, encrypting, and/or applying other suitable modifications to portions or all relevant image data.[0074]- The video sanitization system 120 once a region for sanitization is determined and initiated, the image data is preferably transformed and updated within the video storage system 130.)
In regards to claim 3, The combination of Glaser and DeMaio teach the device according to claim 1, wherein where the processor receives a request from a third party to transmit the first image to the remote data storage, the processor is configured to:check if the third party obtains a predetermined approval(US 20210035342, Glaser, para. 0085, The image data may alternatively be indirectly collected wherein collecting image data includes receiving image data record by an external imaging system. In this variation, the method may not include the actual sensing of the image data, but facilitates the management and processing of the image data for the purposes of image sanitization[i.e. note: implicit approval is given].), and transmit the first image to the remote data storage if the third party obtains the predetermined approval(US 20210035342, Glaser, para. 0071, Alternatively, user or external input[i.e. note: 3rd party] may be used in at least partially determining a sanitization region.). In regards to claim 4, The combination of Glaser and DeMaio teach the device according to claim 1, wherein the processor is configured to:detect if the device is located within a predetermined geofence(US 20210035342, Glaser, para. 0128, a set of sanitization markers is applied in multiple locations in the physical environment to define a regional perimeter. The sanitization markers may be used to mark a two-dimensional space or a three-dimensional space.), and Glaser does not teach control the image capturing module to stop obtaining the first image if the device is detected to be located within the predetermined geofence l However, DeMaio does not teach control the image capturing module to stop obtaining the first image if the device is detected to be located within the predetermined geofence l(US 11282174, DeMaio, col. 48, lines 5-9, For example, a user could block or unblock all their images from a week of vacation or their trip to a convention or all their images around (i.e., within X radius) a certain geographic location.). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of Glaser with the teaching of DeMaio because a user would have been motivated to enable sharing of personal information, taught by DeMaio, by using profile data to control image accessibility in the system taught by Glaser(DeMaio, col. 11, lines 31-49)
In regards to claim 5, The combination of Glaser and DeMaio teach the device according to claim 1, wherein the processor is configured to: detect a context of the external environment, and control the image capturing module to stop obtaining the first image if the context of the external environment relates to a predetermined context(US 20210035342, Glaser, para. 0049 and 0120: [0049]- Additionally, modeling of object-permanence and contextual understanding of the observed activity in the image data may be leveraged to appropriately apply image sanitization.[0120]- Content of a display may additionally be classified and used in assigning a sanitization region around the screen. For example, this may include detecting an authentication screen and engaging image sanitization to prevent capture of information related to authentication credentials.). In regards to claim 6, the combination of Glaser and DeMaio teach the device according to claim 1, wherein where the processor receives a notification that at least one of the one or more regions of the second image does not include an object of the predetermined type from a third party, the processor is configured to check whether the predetermined time has passed(US 11282174, DeMaio, col. 20, lines 46-57, In one example, a keyword on website 1 254 might be “Disneyland.” While presenting to the user device 208 that graphical images for website 1 254, the non-approved entity module 225 might present or overlay a graphical notification indicating that there is additional data or rewards-based opportunity available to the user that can be accessed if the user clicks on the graphical notification. The non-approved entity module 225 can provide additional overlay functionality for the authorized user device 208, through the rewards-based, universal, integrated code base 203, when the authorized user device 208 interacts with the unauthorized third-party entity 254, 256.).
In regards to claim 7, the combination of Glaser and DeMaio teach the device according to claim 6, wherein if the predetermined time has not passed, the processor is configured to:check if the third party obtains a predetermined approval, extract the first image from the memory if the third party obtains the predetermined approval, generate another second image corresponding to the extracted first image, without blurring a region of the another second image corresponding to the at least one of the one or more regions of the second image(US 11282174, DeMaio, fig. 8C and col. 45, line 55-col. 46, line 4, FIG. 8C illustrates a selectable object 808 which can be presented on the user interface 800 on the user device 208 of the person who took the picture 804. The selectable object 808 could be related to a manual interaction in which the user is asked whether they should send an unblock request associated with the particular person 806. The user can manually select to request the unblock or to cancel and thus maintain the blocking. While the unblock request can be manual as shown in this figure, the unblock request can also be established to be sent automatically as picture 804 may include many individuals who have their face or body blurred. The system may identify when a blurred image is provided in a picture and then automatically send unblock requests to the identified individuals. Typically, the blurred images are people known to the system and thus the system has the contact information of the blocked people to make the request to unblock possible to send automatically.), and transmit the another second image to the remote data storage(US 11282174, DeMaio, col. 48, lines 34-44, The rewards-based, universal, integrated code base 203 can be configured to interact with a plurality of authorized user devices 208 and to perform operations including receiving, from a first user device operated by a first user, an image of a second user having a second user device, wherein the second user has a user profile with a privacy setting indicating that pictures or video taken of the second user by the first device should be blurred, based on the user profile, blurring the image of the second user taken by the first user device and transmitting a request to the second user device to unblur the image.).
In regards to claim 8, the combination of Glaser and DeMaio teach the device according to claim 6, wherein if the predetermined time has passed, the processor is configured to:generate information about the at least one of the one or more regions of the second image, and use the information as a training instance for training the processor, so that when another first image capturing the external environment is obtained, a region of the another first image corresponding to the at least one of the one or more regions of the second image is not blurred(US 20210035342, Glaser, para. 0125, selecting the sanitization image region can include detecting one or more sanitization markers in the image data and positioning the sanitization image region according to the one or more sanitization markers as shown in FIG. 12. A sanitization marker is preferably a visual marker that is used to at least partially define a sanitization region. A sanitization marker could be a graphical sticker or decal that could be positioned in areas where privacy….Alternatively, the marker may be a detectable graphic wherein a CV model is trained on detecting the marker.).
In regards to claim 9, the combination of Glaser and DeMaio teach the device according to claim 8, wherein the information includes at least one of geocoordinates and 3D coordinates of the at least one of the one or more regions of the second image(US 20210035342, Glaser, para. 0154 and 0156: [0154]- The spatial interpretation of the image data may be a 3D interpretation, a 2.5D interpretation (e.g., a projected depth map of pixel values), a version of foreground/background interpretation, and/or other formats that provide some level of positional information beyond a 2D image of a scene. The spatial interpretation can be used in determining how to dynamically select pixel values of the images (e.g., the sanitization image region) based on spatial positioning of pixels and/or objects. [0156]- generating a spatial interpretation of image data and thereby spatially interpreting image data may include predicting a depth map from image data.).
In regards to claim 10, Glaser teaches a method for processing an image in a device, the method comprising:obtaining a first image capturing an external environment(US 20210035342, Glaser, para. 0079, wherein a method for automatic privacy preservation using CV-based interpretation of the environment can include collecting image data from an environment S310);detecting one or more regions of the first image, wherein the one or more regions include one or more objects of a predetermined type(US 20210035342, Glaser, para. 0159, The type of image sanitization may be selected from a set of possibilities. In one variation, the type of image sanitization depends on the detected object. The application of image sanitization may be applied using basic region geometry (e.g., a blacking out a rectangle thereby covering an object).);generating a second image corresponding to the first image, by blurring one or more regions of the second image each corresponding to the one or more regions of the first image(US 20210035342, Glaser, para. 0030 and 0079: [0030]- A primary example of image sanitization may include image redaction or blurring.[0079]- the method may be adapted to use a form of conditional image sanitization, wherein a method for automatic privacy preservation using CV-based interpretation of the environment can include collecting image data from an environment S310; detecting an object and or event registered for image sanitization S320; selecting the sanitization image region based on location of the object and/or event in the image data S332; applying image sanitization to the sanitization image region thereby generating sanitized image data S334 as shown in FIG. 16. );storing the first image in a memory of the device for a predetermined time(US 20210035342, Glaser, para. 0171, original image data (pre-sanitization) can be stored in a data storage system and then processing of the image data and generating sanitized image data is applied using the stored image data. In this manner, the original version of the image data can be preserved. The sanitized image data may be separately stored, communicated to another system, and/or used to replace the stored image data. In one variation, the sanitized image data may be stored separately for use with particular systems, while the original may be used as a canonical record. In another variation, image data meeting certain conditions may be replaced with sanitized image data, which functions to store partially sanitized and unsanitized image data. For example, image data may be updated to sanitized image data after some set time duration (e.g., 24 hours, 1 week, etc.)); and Glaser does not teach transmitting the second image to a remote data storage via data communication with the remote data storage However, DeMaio teaches transmitting the second image to a remote data storage via data communication with the remote data storage(US 11282174, DeMaio, col. 48, lines 34-44, The rewards-based, universal, integrated code base 203 can be configured to interact with a plurality of authorized user devices 208 and to perform operations including receiving, from a first user device operated by a first user, an image of a second user having a second user device, wherein the second user has a user profile with a privacy setting indicating that pictures or video taken of the second user by the first device should be blurred, based on the user profile, blurring the image of the second user taken by the first user device and transmitting a request to the second user device to unblur the image.). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of Glaser with the teaching of DeMaio because a user would have been motivated to enable sharing of personal information, taught by DeMaio, by using profile data to control image accessibility in the system taught by Glaser(DeMaio, col. 11, lines 31-49) In regards to claim 11, The combination of Glaser and DeMaio teach the method according to claim 10, wherein the storing the first image in the memory comprises: storing the first image in the memory in an encrypted format(US 20210035342, Glaser, para. 0030 and 0074: [0030]- Sanitization can involve fully or partially redacting, reducing resolution/fidelity, modifying, encrypting, and/or applying other suitable modifications to portions or all relevant image data.[0074]- The video sanitization system 120 once a region for sanitization is determined and initiated, the image data is preferably transformed and updated within the video storage system 130.)
In regards to claim 12, The combination of Glaser and DeMaio teach the method according to claim 10 further comprising: where a request to transmit the first image to the remote data storage is received from a third party, checking if the third party obtains a predetermined approval(US 20210035342, Glaser, para. 0085, The image data may alternatively be indirectly collected wherein collecting image data includes receiving image data record by an external imaging system. In this variation, the method may not include the actual sensing of the image data, but facilitates the management and processing of the image data for the purposes of image sanitization[i.e. note: implicit approval is given].); andtransmitting the first image to the remote data storage if the third party obtains the predetermined approval(US 20210035342, Glaser, para. 0071, Alternatively, user or external input[i.e. note: 3rd party] may be used in at least partially determining a sanitization region.). In regards to claim 13, the combination of Glaser and DeMaio teach the method according to claim 10 further comprising:detecting if the device is located within a predetermined geofence(US 20210035342, Glaser, para. 0128, a set of sanitization markers is applied in multiple locations in the physical environment to define a regional perimeter. The sanitization markers may be used to mark a two-dimensional space or a three-dimensional space.); andstopping obtaining the first image if the device is detected to be located within the predetermined geofence(US 11282174, DeMaio, col. 48, lines 5-9, For example, a user could block or unblock all their images from a week of vacation or their trip to a convention or all their images around (i.e., within X radius) a certain geographic location.).
In regards to claim 14, The combination of Glaser and DeMaio teach the method according to claim 10 further comprising:detecting a context of the external environment(US 20210035342, Glaser, para. 0049, Additionally, modeling of object-permanence and contextual understanding of the observed activity in the image data may be leveraged to appropriately apply image sanitization.); andstopping obtaining the first image if the context of the external environment relates to a predetermined context(US 20210035342, Glaser, para. 0120, Content of a display may additionally be classified and used in assigning a sanitization region around the screen. For example, this may include detecting an authentication screen and engaging image sanitization to prevent capture of information related to authentication credentials.). In regards to claim 15, the combination of Glaser and DeMaio teach the method according to claim 10 further comprising: where a notification that at least one of the one or more regions of the second image does not include an object of the predetermined type is received from a third party, checking whether the predetermined time has passed(US 11282174, DeMaio, col. 20, lines 46-57, In one example, a keyword on website 1 254 might be “Disneyland.” While presenting to the user device 208 that graphical images for website 1 254, the non-approved entity module 225 might present or overlay a graphical notification indicating that there is additional data or rewards-based opportunity available to the user that can be accessed if the user clicks on the graphical notification. The non-approved entity module 225 can provide additional overlay functionality for the authorized user device 208, through the rewards-based, universal, integrated code base 203, when the authorized user device 208 interacts with the unauthorized third-party entity 254, 256.).
In regards to claim 16, the combination of Glaser and DeMaio teach the method according to claim 15 further comprising:if the predetermined time has not passed, checking if the third party obtains a predetermined approval(US 11282174, DeMaio, col. 38, lines 52-60, The user might have a platform for mobile orders and a hosted website hosted via the web hosting platform 218I, as well as an approved site 258. The command to replace a picture across these various platforms can be carried out by the data/privacy/file storage management module 236 such that the user does not need to drag and drop, or open up an application, or perform some other traditional operation to activate an application or go to a website to change out an image.);extracting the first image from the memory if the third party obtains the predetermined approval;generating another second image corresponding to the extracted first image, without blurring a region of the another second image corresponding to the at least one of the one or more regions of the second image(US 11282174, DeMaio, fig. 8C and col. 45, line 55-col. 46, line 4, FIG. 8C illustrates a selectable object 808 which can be presented on the user interface 800 on the user device 208 of the person who took the picture 804. The selectable object 808 could be related to a manual interaction in which the user is asked whether they should send an unblock request associated with the particular person 806. The user can manually select to request the unblock or to cancel and thus maintain the blocking. While the unblock request can be manual as shown in this figure, the unblock request can also be established to be sent automatically as picture 804 may include many individuals who have their face or body blurred. The system may identify when a blurred image is provided in a picture and then automatically send unblock requests to the identified individuals. Typically, the blurred images are people known to the system and thus the system has the contact information of the blocked people to make the request to unblock possible to send automatically.); andtransmitting the another second image to the remote data storage(US 11282174, DeMaio, col. 48, lines 34-44, The rewards-based, universal, integrated code base 203 can be configured to interact with a plurality of authorized user devices 208 and to perform operations including receiving, from a first user device operated by a first user, an image of a second user having a second user device, wherein the second user has a user profile with a privacy setting indicating that pictures or video taken of the second user by the first device should be blurred, based on the user profile, blurring the image of the second user taken by the first user device and transmitting a request to the second user device to unblur the image.).
In regards to claim 17, the combination of Glaser and DeMaio teach the method according to claim 15 further comprising: if the predetermined time has passed, generating information about the at least one of the one or more regions of the second image(US 20210035342, Glaser, para. 0171, original image data (pre-sanitization) can be stored in a data storage system and then processing of the image data and generating sanitized image data is applied using the stored image data. In this manner, the original version of the image data can be preserved. The sanitized image data may be separately stored, communicated to another system, and/or used to replace the stored image data. In one variation, the sanitized image data may be stored separately for use with particular systems, while the original may be used as a canonical record. In another variation, image data meeting certain conditions may be replaced with sanitized image data, which functions to store partially sanitized and unsanitized image data. For example, image data may be updated to sanitized image data after some set time duration (e.g., 24 hours, 1 week, etc.)); andusing the information as a training instance for training a processor performing the method, so that when another first image capturing the external environment is obtained, a region of the another first image corresponding to the at least one of the one or more regions of the second image is not blurred(US 20210035342, Glaser, para. 0125, selecting the sanitization image region can include detecting one or more sanitization markers in the image data and positioning the sanitization image region according to the one or more sanitization markers as shown in FIG. 12. A sanitization marker is preferably a visual marker that is used to at least partially define a sanitization region. A sanitization marker could be a graphical sticker or decal that could be positioned in areas where privacy….Alternatively, the marker may be a detectable graphic wherein a CV model is trained on detecting the marker.).
In regards to claim 18, the combination of Glaser and DeMaio teach the method according to claim 17, wherein the information includes at least one of geocoordinates and 3D coordinates of the at least one of the one or more regions of the second image(US 20210035342, Glaser, para. 0154 and 0156: [0154]- The spatial interpretation of the image data may be a 3D interpretation, a 2.5D interpretation (e.g., a projected depth map of pixel values), a version of foreground/background interpretation, and/or other formats that provide some level of positional information beyond a 2D image of a scene. The spatial interpretation can be used in determining how to dynamically select pixel values of the images (e.g., the sanitization image region) based on spatial positioning of pixels and/or objects. [0156]- generating a spatial interpretation of image data and thereby spatially interpreting image data may include predicting a depth map from image data.).
In regards to claim 19, The combination of Glaser and DeMaio teach a data processing apparatus configured to perform the method of claim 10(US 20210035342, Glaser, para. 0053, As shown in FIG. 1, a system for automatic privacy preservation can include a CV monitoring system 110, a video sanitization system 120, and a video storage system 130. The system is preferably implemented within a particular environment. Various forms of image sanitization may be applied within the environment. The system preferably includes a processing system configured to perform the method processes described herein).
In regards to claim 20, The combination of Glaser and DeMaio teach a computer-readable medium comprising program instructions, which, when executed by one or more processors, cause the one or more processors to perform the method of claim 10(US 20210035342, Glaser, para. 0054, A CV monitoring system 110 of a preferred embodiment functions to collect image data and facilitate one or more CV processes. Preferably, the CV monitoring system no is used for detecting objects, monitoring users (e.g., humans), tracking user-object interactions, and/or making other conclusions based on image and/or sensor data. The CV monitoring system no will preferably include various computing elements used in processing image data collected by an imaging system.)
CONCLUSION
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GREGORY LANE whose telephone number is (571)270-7469. The examiner can normally be reached on 571 270 7469 from 8:00 AM to 6:00 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Taghi Arani, can be reached on 571 272 3787. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/GREGORY A LANE/Examiner, Art Unit 2438