DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Preliminary Amendment
This Office Action is responsive to communications filed on 02/25/2025. Claims 1-20 are pending in the instant application. Claims 1, 5, 6 and 7 are independent. Applicants submits that support for the amendments is found in the application and claims as originally filed. Applicant further submits no new matter has been added. An Office Action on the merits follows here below.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02/25/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
The abstract of the disclosure is objected to because the patent abstract should be concise statement that does not recite legal phraseology and should not merely recite an instant claim. Correction is required. See MPEP § 608.01[b].
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f), because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitations are: “an acquisition unit” and “a learning unit” at claim 1.
Such claim limitations are: “an estimation unit” and “an image processing unit” at claim 5.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recites sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 6 and 8 are rejected under 35 U.S.C. 102(a)(1) and/or (a)(2) as being anticipated by Sameer (US 20220108521 A1).
Regarding Claim 1: Sameer discloses a learning device (Refer to para [058]; “To identify the objects or features, the processing module(s) 203 may utilize various algorithms (e.g. computer vision or machine learning algorithms) and computing architectures.”) comprising:
an acquisition unit (Refer to para [042]; “In some embodiments, the UE 109 and/or vehicle 105 may include various sensors for acquiring a variety of different data or information.”) that acquires three-dimensional coordinate values (Refer to para [078]; “In some implementations, geographic features (e.g., two-dimensional or three-dimensional features) may be represented in the geographic database 107 using polygons (e.g., two-dimensional features) or polygon extrusions (e.g., three-dimensional features).”) information on a line-of-sight direction (Refer to para [034]; “In general, the system 100 may be any device, apparatus, system, or a combination thereof, that is configured to carry out steps for generating line-of-sight information.”) and point cloud data (Refer to para [046]; “In one embodiment, the services platform 113 may use the output of the data analysis system 103 (e.g., ground control point data) to localize the vehicle 105 or UE 109 (e.g., a portable navigation device, smartphone, portable computer, tablet, etc.), and provide services such as navigation, mapping, other location-based services, and so forth.”) as input data and images captured from a plurality of directions as teacher data (Refer to para [093]; “In some implementations, the HD mapping data records 511 may be created from high-resolution 3D mesh or point-cloud data generated, for instance, from LiDAR-equipped vehicles. The 3D mesh or point-cloud data may be processed to create 3D representations of a street or geographic environment at centimeter-level accuracy for storage in the HD mapping data records 511.”) and a learning unit that learns a model for outputting an image from a designated line-of-sight direction by outputting a color and a density for each pixel using the input data and the teacher data (Refer to para [077]; “In particular, the HD mapping data records 511 may include a variety of data, including data with resolution sufficient to provide centimeter-level or better accuracy of map features. For example, the HD mapping data may include data captured using LiDAR, or equivalent technology capable large numbers of 3D points, and modelling road surfaces and other map features down to the number lanes and their widths. In one embodiment, the HD mapping data (e.g., HD data records 511) capture and store details such as the slope and curvature of the road, lane markings, roadside objects such as signposts, including what the signage denotes. By way of example, the HD mapping data enable highly automated vehicles to precisely localize themselves on the road.”).
Regarding Claim 6: Sameer discloses a learning method (Refer to para [001]; “The present disclosure relates generally to image processing and mapping applications and services, and more specifically to systems and methods for generating line-of-sight information using imagery.”) in which a processor (Refer to para [007]; “In accordance with aspect of the disclosure, a system for generating light-of-sight information using imagery is provided. The system includes at least one processor and at least one memory comprising instructions executable by the at least one processor.”) executes processing of: acquiring three-dimensional coordinate values (Refer to para [078]; “In some implementations, geographic features (e.g., two-dimensional or three-dimensional features) may be represented in the geographic database 107 using polygons (e.g., two-dimensional features) or polygon extrusions (e.g., three-dimensional features).”) information on a line-of-sight direction (Refer to para [034]; “In general, the system 100 may be any device, apparatus, system, or a combination thereof, that is configured to carry out steps for generating line-of-sight information.”) and point cloud data (Refer to para [046]; “In one embodiment, the services platform 113 may use the output of the data analysis system 103 (e.g., ground control point data) to localize the vehicle 105 or UE 109 (e.g., a portable navigation device, smartphone, portable computer, tablet, etc.), and provide services such as navigation, mapping, other location-based services, and so forth.”) as input data and images captured from a plurality of directions as teacher data (Refer to para [093]; “In some implementations, the HD mapping data records 511 may be created from high-resolution 3D mesh or point-cloud data generated, for instance, from LiDAR-equipped vehicles. The 3D mesh or point-cloud data may be processed to create 3D representations of a street or geographic environment at centimeter-level accuracy for storage in the HD mapping data records 511.”) and learning a model for outputting an image from a designated line-of-sight direction by outputting a color and a density for each pixel using the input data and the teacher data (Refer to para [077]; “In particular, the HD mapping data records 511 may include a variety of data, including data with resolution sufficient to provide centimeter-level or better accuracy of map features. For example, the HD mapping data may include data captured using LiDAR, or equivalent technology capable large numbers of 3D points, and modelling road surfaces and other map features down to the number lanes and their widths. In one embodiment, the HD mapping data (e.g., HD data records 511) capture and store details such as the slope and curvature of the road, lane markings, roadside objects such as signposts, including what the signage denotes. By way of example, the HD mapping data enable highly automated vehicles to precisely localize themselves on the road.”).
Regarding Claim 8: Sameer discloses a computer program for causing a computer to function as the learning device according to claim 1 (Refer to para [008]; “In accordance with yet another aspect of the disclosure, a non-transitory computer-readable storage medium, carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to perform steps to receive a raster image depicting at least one object in a region of interest, and measure a shadow of the at least one object in the raster image.”).
Claims 5, 7 and 9 are rejected under 35 U.S.C. 102(a)(1) and/or (a)(2) as being anticipated by Yoon (US 20230419557 A1).
Regarding Claim 5: Yoon discloses an image processing device (“… point cloud data transmission device 10000” at para [057]) comprising: an estimation unit (Refer to para [230]; “In estimating a color value of each point constituting the point cloud, the geometry previously obtained through the smoothing process may be used. In the smoothed point cloud, the positions of some points may have been shifted from the original point cloud, and accordingly a recoloring process of finding colors suitable for the changed positions may be required. Recoloring may be performed using the color values of neighboring points. For example, as shown in the figure, a new color value may be calculated in consideration of the color value of the nearest neighboring point and the color values of the neighboring points.”) that inputs a line-of-sight direction to a learned model for outputting an image from a designated line-of-sight direction by outputting a color and a density for each pixel using three-dimensional coordinate values, information on a line-of-sight direction (Refer to para [084]; “In the acquisition process, data of 3D positions (x, y, z)/attributes (color, reflectance, transparency, etc.) of multiple points, for example, a polygon file format (PLY) (or the Stanford Triangle format) file may be generated. For a video having multiple frames, one or more files may be acquired. During the capture process, point cloud related metadata (e.g., capture related metadata) may be generated.”) and point cloud data (Refer to para [064]; “A point cloud data reception device 10005 according to the embodiments may include a receiver 10006, a file/segment decapsulation module 10007, a point cloud video decoder 10008, and/or a renderer 10009.”) as input data and images captured from a plurality of directions as teacher data (Refer to para [065]; “The receiver 10006 according to the embodiments receives a bitstream containing point cloud video data. According to embodiments, the receiver 10006 may transmit feedback information to the point cloud data transmission device 10000.”) and causes the model to output a color and a transmittance for each pixel from the line-of-sight direction (Refer to para [368]; “In order to compress input 3D point cloud data, the V-PCC method generates, from the input data, three video streams: 1) an occupancy map video stream, 2) a geometry video stream, and 3) an attribute video stream. They are compressed respectively using a 2D video codec in the V-PCC encoder. Among the videos, the attribute video is generated to contain the color attribute of a 3D point in the 2D image. When the attribute video has texture type information, each pixel value in the image represents a (R, G, B) color attribute of corresponding 3D point.”) and an image processing unit (Refer to para [113]; “A unit may represent a basic unit of image processing. The unit may include at least one of a specific region of the picture and information related to the region.”) that generates an image from the line-of-sight direction using the color and the transmittance output by the estimation unit (Refer to para [230]; “In estimating a color value of each point constituting the point cloud, the geometry previously obtained through the smoothing process may be used. In the smoothed point cloud, the positions of some points may have been shifted from the original point cloud, and accordingly a recoloring process of finding colors suitable for the changed positions may be required. Recoloring may be performed using the color values of neighboring points. For example, as shown in the figure, a new color value may be calculated in consideration of the color value of the nearest neighboring point and the color values of the neighboring points.”).
Regarding Claim 7: Yoon discloses an image processing method in which a processor executes processing of (Refer to para [004]; “An object of the present disclosure is to provide a point cloud data transmission device, a point cloud data transmission method, a point cloud data reception device, and a point cloud data reception method for efficiently transmitting and receiving a point cloud.”): inputting a line-of-sight direction to a learned model for outputting an image from a designated line-of-sight direction by outputting a color and a density for each pixel using three-dimensional coordinate values (Refer to para [084]; “In the acquisition process, data of 3D positions (x, y, z)/attributes (color, reflectance, transparency, etc.) of multiple points, for example, a polygon file format (PLY) (or the Stanford Triangle format) file may be generated. For a video having multiple frames, one or more files may be acquired. During the capture process, point cloud related metadata (e.g., capture related metadata) may be generated.”) information on a line-of-sight direction, and point cloud data (Refer to para [064]; “A point cloud data reception device 10005 according to the embodiments may include a receiver 10006, a file/segment decapsulation module 10007, a point cloud video decoder 10008, and/or a renderer 10009.”) as input data and images captured from a plurality of directions as teacher data (Refer to para [065]; “The receiver 10006 according to the embodiments receives a bitstream containing point cloud video data. According to embodiments, the receiver 10006 may transmit feedback information to the point cloud data transmission device 10000.”) and causing the model to output a color and a transmittance for each pixel from the line-of-sight direction (Refer to para [368]; “In order to compress input 3D point cloud data, the V-PCC method generates, from the input data, three video streams: 1) an occupancy map video stream, 2) a geometry video stream, and 3) an attribute video stream. They are compressed respectively using a 2D video codec in the V-PCC encoder. Among the videos, the attribute video is generated to contain the color attribute of a 3D point in the 2D image. When the attribute video has texture type information, each pixel value in the image represents a (R, G, B) color attribute of corresponding 3D point.”) and generating an image from the line-of-sight direction using the color and the transmittance (Refer to para [230]; “In estimating a color value of each point constituting the point cloud, the geometry previously obtained through the smoothing process may be used. In the smoothed point cloud, the positions of some points may have been shifted from the original point cloud, and accordingly a recoloring process of finding colors suitable for the changed positions may be required. Recoloring may be performed using the color values of neighboring points. For example, as shown in the figure, a new color value may be calculated in consideration of the color value of the nearest neighboring point and the color values of the neighboring points.”).
Regarding Claim 9: Yoon discloses a computer program for causing a computer to function as the image processing device according to claim 5 (Refer to para [618]; “Operations according to embodiments described in the present disclosure may be performed by a transmission/reception device including a memory and/or a processor according to embodiments. The memory may store programs (flow charts, etc.) for processing/controlling operations according to embodiments, and the processor may control various operations described in this document. The processor may be referred to as a controller or the like. The operations in the embodiments may be performed by firmware, software, and/or a combination thereof. The firmware, software, and/or combination thereof may be stored in the processor or the memory.”).
Allowable Subject Matter
Claims 2-4 and 10-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The prior art either singly or in combination does not teach, disclose or suggest at least the following claim limitation(s): “… the learning unit learns the model so as to output the density for each pixel by inputting a first feature amount obtained from the point cloud data and the three-dimensional coordinate values to a predetermined first neural network, and to output the color for each pixel by inputting a feature amount obtained from the information on the line-of-sight direction and the first feature amount to a predetermined second neural network.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Nagano (US 20220005212 A1)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MIA M THOMAS whose telephone number is (571)270-1583. The examiner can normally be reached M-Th 8:30am-4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen (Steve) Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MIA M. THOMAS
Primary Examiner
Art Unit 2665
/MIA M THOMAS/Primary Examiner
Art Unit 2665
Prosecution Insights