DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 14, and therefore claims 15–20 which depend therefrom, are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims do not fall within at least one of the four categories of patent eligible subject matter because it is directed towards a “A computer program product comprising: one or more computer-readable storage media,” which, under a broadest reasonable interpretation, does not require so narrow of an interpretation as being a “non-transitory computer-readable storage media” and thus includes signals per se which are non-statutory subject matter. The Examiner notes that the originally filed specification in paragraph 36 recites “A computer-readable storage medium, as that term is used in the present disclosure, is not to be construed as storage in the form of transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through a fiber optic cable, electrical signals communicated through a wire, and/or other transmission media” however, such a statement does not constitute a clear disavowal of scope, as required for applicant to be their own lexicographer and redefine the plain meaning of “computer-readable storage media” which has been determined by the Federal Circuit to include transitory signals. See MPEP §2111.01 (IV) and §2106(II). Specifically, claim construction “instructions” to the Examiner do not constitute a clear definition of the terms “computer-readable storage media.” Therefore, claims 14–20 are directed to non-statutory subject matter and is rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1–2, 6–9, 12–15 and 19–20 are rejected under 35 U.S.C. 103 as being unpatentable over Herman et al., US Patent Application Publication No. US 2023/0375707 A1 (herein “Herman”) in view of Robinson et al., US Patent No. 8,316,450 B2 (herein “Robinson”).
Regarding claims 1, 8 and 14, with substantive differences between the claims noted in curly brackets {}, deficiencies of Herman noted in square brackets, and with claim 1 as exemplary, Herman teaches {a method comprising: - claim 1 / A smart vehicle system comprising: a processor set; one or more computer-readable storage media; and program instructions stored on the one or more computer-readable storage media to cause the processor set to perform operations comprising: - claim 8 / A computer program product comprising: one or more computer-readable storage media; and program instructions stored on the one or more computer-readable storage media to perform operations comprising: - claim 14} (Herman Abstract, a computer including a processor and a memory storing instructions to execute by the processor, lidar data processing steps)
identifying a plurality of relevant points on each of one or more of a set of objects captured in an image of an environment needing to be abstracted (Herman ¶¶ 37–38, 40–43 and 48, lidar data captured from a vehicle computer and from which a point cloud is generated, the point cloud including sensitive personally identifiable information PII (needing to be abstracted) such as a license plate (one or more of a set of objects) comprised of specific points in the point cloud, where the points belonging to the PII objects have a point density that is reduced, and where the reduced density point cloud is transmitted to a remote computer for further processing, thus remaining points from the reduced density of the PII object being identified as relevant);
[inserting] the plurality of relevant points corresponding to each of the one or more of the set of objects into [the image] of the environment forming an abstraction of each of the one or more of the set of objects needing to be abstracted (Herman ¶¶40 and 43, fig. 2, while reducing the density of the point cloud by deleting some points in a region 210 corresponding to an object 205 with PII, thus leaving only certain points (relevant points), the density and points of the point cloud areas other than region 210 such as region 215 that do not have PII, including points surrounding and outside of region 210 (the environment) retained/unchanged); and
masking details in [the image] of the environment except for the plurality of relevant points corresponding to each of the one or more of the set of objects inserted into [the image] to form an abstracted [image] of each of the one or more of the set of objects in the environment to preserve confidentiality (Herman ¶¶ 44 and 41, a geometric blurring or random noise (both which mask details) is applied to a subset of the points in the region 210, leaving alone (except for) the remaining points that meet the desired threshold density for ensuring that the resulting density is not enough to identify a person or perform text recognition (preserve confidentiality)).
While Herman discloses its point cloud abstraction process to retain surrounding environment points in a point cloud and simply to delete some of the points in the PII object “needing to be abstracted,” in such a process, while the end result of having a point cloud where only relevant points for objects needing abstraction are present, nonetheless, Herman does not explicitly teach “inserting” the relevant points into “the image” of the environment. Further, while Herman discloses lidar based point clouds, Herman does not explicitly teach images, although.
However, Robinson teaches inserting graphical data (which would include pixels/points) of an object into an image of an environment (Robinson col. 4, ll. 60-65, col. 21, l. 60 – col. 22, l. 11, fig. 14e, objects as images inserted at a marker in the content, where the content is an image of an environment).
Therefore, taking the teachings of Herman and Robinson as a whole, it would have been obvious to a person having ordinary skill in the art (herein “PHOSITA”) before the effective filing date of the claimed invention to have modified the point cloud visual data density reduction for keeping secure PII method of Herman with the insertion of objects data into an environmental image disclosed in Robinson at least because doing so would allow for a connection between consumers of image data specific to certain objects, thus providing exchange with other users in mutual/shared activities. See Robinson col. 3, ll. 29–35, and col. 22, ll. 12–21.
Regarding claims 2, 9 and 15, with claim 2 as exemplary, and deficiencies of Herman noted in square brackets, Herman teaches further comprising: [attaching contextual information regarding the environment as metadata to the] abstracted [image of each of the one or more of the set of objects in the environment]; and sending the abstracted [image] of each of the one or more of the set of objects in the environment with the contextual information regarding the environment [attached as the metadata] to a set of data analysis services (Herman ¶¶ 5, 28, 48 and 51, a transceiver on the vehicle provides communication from the vehicle to a remote computer 110, the communication including the lidar data with the specific to PII object region reduced (abstracted) point cloud data to the remote computer 110 for further analysis such as performance of an advanced driver assistance systems (ADAS) feature of the vehicle).
Herman does not specifically teach where Robinson teaches attaching contextual information regarding the environment as metadata to the image of each of the one or more of the set of objects in the environment; and contextual information regarding the environment attached as the metadata (Robinson col. 3, ll. 62–65, col. 4, ll. 30–44, col. 17, ll.9–50, claim 1, metadata describing markers, objects and content are part of the data and processed for the markers, objects and content, an thus when an object is downloaded and inserted into content, the metadata is attached thereto, the metadata including data corresponding to one or both of a temporal and spatial positioning of the marker within a viewer perceivable portion of the content (thus contextual regarding the environment)).
Therefore, taking the teachings of Herman and Robinson as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the point cloud visual data density reduction for keeping secure PII method of Herman with the insertion of objects data including metadata into an environmental image as disclosed by Robinson at least because doing so would allow for a connection between consumers of image data specific to certain objects, thus providing exchange with other users in mutual/shared activities. See Robinson col. 3, ll. 29–35, and col. 22, ll. 12–21.
Regarding claims 6, 12 and 19, with claim 6 as exemplary, Herman teaches further comprising: determining whether the image of the environment captures the set of objects based on performing an analysis of the image (Herman ¶¶37–40, object recognition on the point cloud data fused with camera data (images) is performed and a CNN outputs the detected object type and whether that object type is associated with PII); and responsive to determining that the image of the environment does capture the set of objects based on the analysis of the image, applying a set of confidentiality criteria to the set of objects captured in the image of the environment (Herman ¶¶40–41, objects with PII are then evaluated to see if the point cloud density is above a certain threshold associated with being able to discern the PII (confidentiality criteria)).
Regarding claims 7, 13 and 20, with claim 7 as exemplary, Herman teaches further comprising: determining whether the one or more of the set of objects captured in the image of the environment need to be abstracted based on applying a set of confidentiality criteria (Herman ¶¶40–43, objects with PII are then evaluated to see if the point cloud density is above a certain threshold associated with being able to discern the PII (confidentiality criteria), and of they are, then the point cloud is processed to reduce the density in the region where the object having PII exists); and responsive to determining that the one or more of the set of objects captured in the image of the environment do need to be abstracted based on applying the set of confidentiality criteria, identifying the plurality of relevant points on each of the one or more of the set of objects captured in the image of the environment needing to be abstracted (Herman ¶¶42–43, the density of the point cloud in the region with the object having PII is reduced, therefore the remaining points in the point cloud region being identified as relevant points).
Claims 3–5, 10–11 and 16–18 are rejected under 35 U.S.C. 103 as being unpatentable over Herman in view of Robinson, as set forth above regarding the independent claims, and further in view of Ravella et al., US Patent Application Publication No. 2025/0124794 A1 (herein “Ravella”).
Regarding claims 3, 10 and 16, with deficiencies of Herman noted in square brackets [] and with claim 3 as exemplary, Herman teaches further comprising: [receiving information regarding] analysis of the abstracted [image] of each of the one or more of the set of objects in the environment surrounding a smart vehicle [with the contextual information regarding the environment attached as the metadata from the set of data analysis services] (Herman ¶¶51–56, lidar data relating to the point cloud of objects with PII that has been reduced in point density for those objects is saved and used for further analysis including automatic driving assistance for the vehicle); and
operating functional components of the smart vehicle automatically based on the information regarding the analysis of the abstracted image of each of the one or more of the set of objects in the environment surrounding the smart vehicle (Herman ¶51, further analysis for performance of an ADAS (operating functional components of the smart vehicle automatically) feature of the vehicle and for a particular object in the lidar data) [with the contextual information regarding the environment attached as the metadata received from the set of data analysis services].
As noted above, Herman does not, however Robinson at least teaches with the contextual information regarding the environment attached as the metadata (Robinson col. 3, ll. 62–65, col. 4, ll. 30–44, col. 17, ll.9–50, claim 1, metadata describing markers, objects and content are part of the data and processed for the markers, objects and content, an thus when an object is downloaded and inserted into content, the metadata is attached thereto, the metadata including data corresponding to one or both of a temporal and spatial positioning of the marker within a viewer perceivable portion of the content (thus contextual regarding the environment)).
Further, although Herman teaches for example in ¶51 that the abstracted data can be used for further analysis and stored by the remote computer for automated driving assistance in the vehicle, nonetheless Herman does not explicitly teach that the remote compute sends analysis information back to the vehicle, and therefore Herman does not explicitly teach where Ravella teaches receiving information regarding analysis of the image from the set of data analysis services (Ravella fig. 1, ¶¶58–59, 69, 139, 197, 205–207, server 104 collects and analyzes vehicle data, including data from cameras (images), and provides driving solutions to individual vehicles 102 including vehicle control, as well as performing the operations of computer system 600, including providing to vehicles via a wireless network, trajectory data for the vehicle from analyzing the sensor data).
Therefore, taking the teachings of Herman and Robinson as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the point cloud visual data density reduction for keeping secure PII method of Herman with the insertion of objects data including metadata into an environmental image as disclosed by Robinson at least because doing so would allow for a connection between consumers of image data specific to certain objects, thus providing exchange with other users in mutual/shared activities. See Robinson col. 3, ll. 29–35, and col. 22, ll. 12–21.
Further, taking the teachings of Herman as modified by Robinson and Ravella altogether as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the advanced drive assistance system of Herman with the analysis and trajectory information transmitted back to a vehicle from the server as disclosed in Ravella at least because doing so would improve decision making in autonomous vehicles. See Ravella ¶5.
Also further regarding claim 10, and regarding claims 4 and 17, Herman does not explicitly teach, where Ravella teaches wherein the information includes a prediction regarding movement of each of the one or more of the set of objects in the environment surrounding the smart vehicle in relation to speed and direction of movement of the smart vehicle (Ravella ¶¶ 205–213, the travel trajectory to the vehicle sent by the computer system/server includes scene perception and prediction regarding other vehicles in a zone of interest, including other vehicles coming at high speed from the back of the vehicle (spend and direction of movement relative to the smart vehicle)).
Therefore, taking the teachings of Herman as modified by Robinson and Ravella altogether as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the advanced drive assistance system of Herman with the analysis and trajectory information transmitted back to a vehicle from the server as disclosed in Ravella at least because doing so would improve decision making in autonomous vehicles. See Ravella ¶5.
Regarding claims 5, 11 and 18, Herman does not explicitly teach but Ravella teaches further comprising: capturing the image of the environment surrounding the smart vehicle along with contextual information regarding the environment using an Internet of Things sensor set (Ravella ¶¶54–58, each vehicle having sensors including global positioning sensors, lidar scanner, radar sensor, IMU sensors reporting acceleration, orientation, angular rates, odometry sensor, and one or more cameras (capturing an image) of the environment around the vehicle, where the vehicle data is provided to servers, where ¶71 teaches the vehicle (and thus its sensors) communicating with the server via the Internet, thus the sensors being “Internet of Things” sensors, also ¶145 installation sensors providing data to the vehicle on information pertaining to an intersection the vehicle is present within), the contextual information includes time of day (Ravella ¶147, intersection information including the time at which the vehicle entered the current mode), geographic location (Ravella ¶54, GPS sensors provide a geographical position of the vehicle), vehicle speed, and vehicle direction of movement of the smart vehicle (Ravella ¶69, accelerometers and gyroscopes to measure the position, the orientation, and the speed of the vehicle, and also see ¶¶147-148, intersection information including a lane travel direction for the lane the vehicle is in); and performing an analysis of the image of the environment surrounding the smart vehicle using computer vision and a set of machine learning models (Ravella ¶61, image data collected by the cameras is processed using an object detection model, and ¶71 machine learning models processing the vehicle data).
Therefore, taking the teachings of Herman as modified by Robinson and Ravella altogether as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the advanced drive assistance system of Herman with the sensors and data and processing models used as disclosed in Ravella at least because doing so would improve decision making in autonomous vehicles. See Ravella ¶5.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Anders et al., US PGPub No. 2019/0371065 A1, directed towards modifying an augmented reality scene, by generating masks for objects in the scene including a vehicle.
Speciale et al., "Privacy Preserving Image-Based Localization," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 5488-5498, doi: 10.1109/CVPR.2019.00564, directed towards a way to abstract images via selecting points in a point cloud and transforming them into line representations.
Forster et al., "Decoding DOOH Viewability using YOLO for Privacy-Friendly Human Silhouette Identification on LiDAR Point Clouds," 2024 12th International Symposium on Digital Forensics and Security (ISDFS), San Antonio, TX, USA, April 30, 2024, pp. 1-6, doi: 10.1109/ISDFS60797.2024.10527261, directed towards privacy friendly human identification from lidar point clouds.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE M KOETH whose telephone number is (571)272-5908. The examiner can normally be reached Monday-Thursday, 09:00-17:00, Friday 09:00-13:00, EDT/EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MICHELLE M. KOETH
Primary Examiner
Art Unit 2671
/MICHELLE M KOETH/Primary Examiner, Art Unit 2671
Prosecution Insights