DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 4 and 10 are objected to because of the following informalities: they recite “the first data set” and its antecedent basis is unclear. Independent claim 1 has introduced “a data set”, which is different from “a first data set.” Appropriate correction is required. One possible amendment is to change Claim 1’s “a data set” to “a first data set.”
Claim 10 is objected to because of the following informalities: the claim recites “the first data set comprises having a first data set that is geo-registered.” Please clarify how “the first data set comprises having a first data set that is geo-registered” is differentiated from “the first data set is geo-registered.” It is unclear whether “the first data set” and “a first data set” refer to the same data set. If they are, it seems redundant. Appropriate correction or clarification is required.
Claim 11 is objected to because of the following informalities: the claim recites “the geo-registered data set”, and the antecedent basis appears unclear. Claim 10 recites “a first data set that is geo-registered”; however, Claim 11 depends on Claim 1. Appropriate correction or clarification is required.
Claim 15 is objected to because of the following minor informalities: The term “simply” in the last line of the claim should be changed to “simplify.” Appropriate correction is required.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the claims at issue are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the reference application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/forms/. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to http://www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1-5, 10-11, and 13-15 are rejected on the ground of nonstatutory double patenting as being unpatentable over Claims 1, 3-6, and 10-13 of US 11,403,822, even though the claims at issue are not identical. The Examiner bolded conflicting patent’s relevant limitations and added (a), (b), and (c) to show correspondence between the conflicting claims.
Instant Application 19/212,355
US 11,403,822
1. A method, comprising:
(a) receiving a data set;
(b) converting the data set to a common reference frame; and
(c) generating a virtual object based on the data set in the common reference frame.
1. A method for generating a guide line comprising:
(a) receiving a data set, wherein the data set is missing information in order to render a virtual object in three dimensions;
storing a reference data set in memory, wherein the reference data set is stored in a mesh at different mesh densities and comprises information about the missing information; supplying the missing information from the data set with data from the reference data set; providing an augmented reality system within a real world environment;
defining a common reference frame based on a local reference frame related to the real world environment in relation to the augmented reality system residing in the real world environment;
(b) converting the data set to the common reference frame; and
(c) generating the virtual object based on the data set in the common reference frame on an augmented reality system such that the virtual object is rendered in relation to the real world environment directly viewable through the augmented reality system in which the virtual object is overlaid.
19/212,355
1-4
5
10
11
13
14
15
US 11,403,822
1
10
3
11
12
12
13
Notes:
Regarding Claim 4’s second data set, it is mapped to US 11,403,822’s reference data.
US 11,403,822’s Claim 10 depends on Claims 1 and 3-6. Therefore, Claim 5 of the instant application is rejected based on Claims 1, 3-6, and 10 of US 11,403,822.
US 11,403,822’s Claim 11 depends on Claim 10.
Claims 1 and 11-12 are rejected on the ground of nonstatutory double patenting as being unpatentable over Claims 1 and 5 of US 12,340,470, even though the claims at issue are not identical. The Examiner bolded conflicting patent’s relevant limitations and added (a), (b), and (c) to show correspondence between the conflicting claims.
Application 19/212,355
US 12,340,470
1. A method, comprising:
(a) receiving a data set;
(b) converting the data set to a common reference frame; and
(c) generating a virtual object based on the data set in the common reference frame.
1. A method, comprising:
(a) receiving a first data set at an augmented reality headset comprising geolocation information of a plurality of vessels;
receiving a second data set in real time at the augmented reality headset comprising information about the vessels;
using the first data set, determining an updated first data set comprising a present location of the plurality of vehicles;
(b) converting the updated first data set and the second data set to a common reference frame relative to the augmented reality headset; and
(c) generating a plurality of virtual objects on the augmented reality headset, each virtual object of the plurality of virtual objects corresponding to each vessel of the plurality of vessels based on the updated first data set and the second data set in the common reference frame to render each virtual object on the augmented reality headset in real time at a displayed location when viewed through the augmented reality headset in relation to the present location of each vessel.
19/212,355
1
11
12
US 12,340,470
1
5
1
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-2, 4-10, and 13-15 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception without significantly more.
Claim 1
Step 1: Claim 1 is directed to a method, thereby meeting step 1.
Step 2A, Prong One: Claim 1 recites a “mental process” abstract idea that can be performed in the human mind or by using a pen and paper and/or a mathematical concept.
converting the data set to a common reference frame; and
Note, under BRI, a common reference frame does not have to be a rendered image, and could be interpreted as or including encoded data according to shared encoding framework.
According to MPEP, citing a case law, "Adding one abstract idea (math) to another abstract idea (encoding and decoding) does not render the claim non-abstract." MPEP 2106.04.II.A.
Further, MPEP 2106.04(a)(2) provided examples for limitations that may be practically performed in the human mind, which includes an example that is similar to the instant application. MPEP states, “a claim to ‘collecting information, analyzing it, and displaying certain results of the collection and analysis,’ where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016).”
Step 2A, Prong Two: The following additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
(a) receiving a data set;
(b) generating a virtual object based on the data set in the common reference frame.
Note, under BRI, a “virtual object” could include displayed raw data, which is not a physical object in a physical environment.
The additional element (a) is insignificant extra-solution activity, specifically, mere data gathering. MPEP 2106.05(g)(3), “Mere Data Gathering” examples iii, iv. “Adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g).” MPEP 2106.04(d) Integration of a Judicial Exception Into A Practical Application.
Regarding (b), the additional element of generating/displaying/outputting step is insignificant extra-solution activity, specifically, selecting a particular data source or type of data to be manipulated. The generating/displaying/outputting step is similar to the MPEP 2106.05(g)(3), “Selecting a particular data source or type of data to be manipulated” examples iii. The generating/displaying/outputting step amounts “necessary data gathering and outputting.” MPEP 2106.05(g)(3).
Also note, the claim does not even requires the use of computer.
Step 2B: Additional elements are determined not to amount to an inventive concept after having considered them both individually and in combination; and the additional elements do not amount to significantly more than the judicial exception itself.
(a) receiving a data set;
(b) generating a virtual object based on the data set in the common reference frame.
The additional element (a) is insignificant extra-solution activity, specifically, mere data gathering. MPEP 2106.05(g)(3), “Mere Data Gathering” examples iii, iv. The additional element is nominal or tangential addition to the claim. MPEP 2106.05(d)II examples i, iv.
Regarding (b), the additional element of generating/displaying/outputting step is insignificant extra-solution activity, specifically, selecting a particular data source or type of data to be manipulated. The generating/displaying/outputting step is similar to the MPEP 2106.05(g)(3), “Selecting a particular data source or type of data to be manipulated” examples iii. The generating/displaying/outputting step amounts “necessary data gathering and outputting.” MPEP 2106.05(g)(3). “Presenting Offers” and “. . . requiring a consumer to view an advertisement” MPEP 2106.05(d).II.
Therefore, Claim 1 is rejected under 35 U.S.C. 101 for being directed to an abstract idea without significantly more.
Claim 2
Step 1: Claim 2 depends on Claim 1 and is directed to a method that is a process, thereby meeting step 1.
Step 2A, Prong One: Claim 2 does not recite additional limitations related to a judicial exception.
Step 2A, Prong Two; Step 2B: Claim 2 recites following additional elements:
wherein the received data set is missing information in order to render the virtual object in three dimensions.
The additional elements amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (computer aided design). MPEP 2106.04(d)(1); 2106.05(j).
Claim 4
Step 1: Claim 4 depends on Claim 1 and is directed to a method that is a process, thereby meeting step 1.
Step 2A, Prong One: Claim 4 does not recite additional limitations related to a judicial exception.
Step 2A, Prong Two; Step 2B: Claim 4 recites following additional elements:
receiving a second data set relating to a physical environment, wherein the first data set comprises missing information and the second data set comprises information related to the missing information of the first data set.
The additional element is insignificant extra-solution activity, specifically, mere data gathering. MPEP 2106.05(g)(3), “Mere Data Gathering” examples iii, iv. The additional element is nominal or tangential addition to the claim. MPEP 2106.05(d)II examples i, iv.
Claim 5
Step 1: Claim 5 depends on Claim 4 and is directed to a method that is a process, thereby meeting step 1.
Step 2A, Prong One: Claim 5 does not recite additional limitations related to a judicial exception.
Step 2A, Prong Two; Step 2B: Claim 5 recites following additional elements:
wherein the second data set comprises elevation of a ground level of the physical environment.
The additional elements amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (computer aided design). MPEP 2106.04(d)(1); 2106.05(j).
Claim 6
Step 1: Claim 6 depends on Claim 5 and is directed to a method that is a process, thereby meeting step 1.
Step 2A, Prong One: Claim 6 recites additional limitations related to a judicial exception: a “mental process” abstract idea that can be performed in the human mind or by using a pen and paper and/or a mathematical concept.
converting the second data set into the common reference frame.
Step 2A, Prong Two; Step 2B: Claim 6 does not recite additional elements.
Claim 7
Step 1: Claim 7 depends on Claim 6 and is directed to a method that is a process, thereby meeting step 1.
Step 2A, Prong One: Claim 7 recites additional limitations related to a judicial exception: a “mental process” abstract idea that can be performed in the human mind or by using a pen and paper and/or a mathematical concept.
simplifying the second data set by reducing a resolution of the data set.
Step 2A, Prong Two; Step 2B: Claim 7 does not recite additional elements.
Claim 8
Step 1: Claim 8 depends on Claim 7 and is directed to a method that is a process, thereby meeting step 1.
Step 2A, Prong One: Claim 8 recites additional limitations related to a judicial exception: a “mental process” abstract idea that can be performed in the human mind or by using a pen and paper and/or a mathematical concept.
generating a mesh mapping of ground level elevation based on the second data set.
Step 2A, Prong Two; Step 2B: Claim 8 does not recite additional elements.
Claim 9
Step 1: Claim 9 depends on Claim 8 and is directed to a method that is a process, thereby meeting step 1.
Step 2A, Prong One: Claim 9 does not recite additional limitations related to a judicial exception.
Step 2A, Prong Two; Step 2B: Claim 9 recites following additional elements:
using the mesh mapping to supply the missing information.
The additional element is insignificant extra-solution activity, specifically, mere data gathering. MPEP 2106.05(g)(3), “Mere Data Gathering” examples iii, iv. The additional element is nominal or tangential addition to the claim. MPEP 2106.05(d)II examples i, iv.
Claim 10
Step 1: Claim 10 depends on Claim 1 and is directed to a method that is a process, thereby meeting step 1.
Step 2A, Prong One: Claim 10 does not recite additional limitations related to a judicial exception.
Step 2A, Prong Two; Step 2B: Claim 10 recites following additional elements:
wherein the receiving the first data set comprises having a first data set that is geo-registered.
The additional element is insignificant extra-solution activity, specifically, mere data gathering. MPEP 2106.05(g)(3), “Mere Data Gathering” examples iii, iv. The additional element is nominal or tangential addition to the claim. MPEP 2106.05(d)II examples i, iv.
Claim 13
Step 1: Claim 13 is directed to a system that comprises one or more processors, a type of machine, thereby meeting step 1.
A system for displaying real-time, geo-registered data in an augmented reality environment, comprising an augmented reality display system, one or more processors, memory, and non-transitory machine readable instructions that when executed by the one or more processors are configured to: . . ..
Step 2A, Prong One: Claim 13 recites a “mental process” abstract idea that can be performed in the human mind or by using a pen and paper or a mathematical concept.
transform the geo-registered information into a data set to be displayed by the augmented reality system; and
Note, under BRI, the data set does not have to be a rendered image, and could be interpreted as or including encoded data according to shared encoding framework.
According to MPEP, citing a case law, "Adding one abstract idea (math) to another abstract idea (encoding and decoding) does not render the claim non-abstract." MPEP 2106.04.II.A.
Further, MPEP 2106.04(a)(2) provided examples for limitations that may be practically performed in the human mind, which includes an example that is similar to the instant application. MPEP states, “a claim to ‘collecting information, analyzing it, and displaying certain results of the collection and analysis,’ where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016).”
Step 2A, Prong Two: The following additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
(a) receive in real time geo-registered information;
(b) display a virtual object on the augmented reality display system based on the real time geo- registered information; displaying real-time, geo-registered data in an augmented reality environment (preamble, if given patentable weight)
(c) “augmented reality display system” and “A system . . . , comprising an augmented reality display system, one or more processors, memory.”
The additional element (a) is insignificant extra-solution activity, specifically, mere data gathering. MPEP 2106.05(g)(3), “Mere Data Gathering” examples iii, iv. “Adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g).” MPEP 2106.04(d) Integration of a Judicial Exception Into A Practical Application.
Regarding (b), the additional element of displaying step is insignificant extra-solution activity, specifically, selecting a particular data source or type of data to be manipulated. The displaying step is similar to the MPEP 2106.05(g)(3), “Selecting a particular data source or type of data to be manipulated” examples iii. The displaying step amounts “necessary data gathering and outputting.” MPEP 2106.05(g)(3).
Regarding (c), under BRI, could be interpreted as a computer or computer components. The computer is recited at a high level of generality, performing a generic computer function. This generic processor limitation is no more than mere instructions to apply the exception using a generic computer component. MPEP 2106.05(f).
Step 2B: Additional elements are determined not to amount to an inventive concept after having considered them both individually and in combination; and the additional elements do not amount to significantly more than the judicial exception itself.
(a) receive in real time geo-registered information;
(b) display a virtual object on the augmented reality display system based on the real time geo- registered information.
(c) “augmented reality display system” and “A system . . . , comprising an augmented reality display system, one or more processors, memory.”
The additional element (a) is insignificant extra-solution activity, specifically, mere data gathering. MPEP 2106.05(g)(3), “Mere Data Gathering” examples iii, iv. The additional element is nominal or tangential addition to the claim. MPEP 2106.05(d)II examples i, iv.
Regarding (b), the additional element of displaying step is insignificant extra-solution activity, specifically, selecting a particular data source or type of data to be manipulated. The displaying step is similar to the MPEP 2106.05(g)(3), “Selecting a particular data source or type of data to be manipulated” examples iii. The displaying step amounts “necessary data gathering and outputting.” MPEP 2106.05(g)(3). “Presenting Offers” and “. . . requiring a consumer to view an advertisement” MPEP 2106.05(d).II.
Regarding (c), under BRI, could be interpreted as a computer or computer components. The computer is recited at a high level of generality, performing a generic computer function. This generic processor limitation is no more than mere instructions to apply the exception using a generic computer component. MPEP 2106.05(f).
Therefore, Claim 13 is rejected under 35 U.S.C. 101 for being directed to an abstract idea without significantly more.
Claim 14
Step 1: Claim 14 depends on Claim 13 and is directed to a machine, thereby meeting step 1.
Step 2A, Prong One: Claim 14 does not recite additional limitations related to a judicial exception.
Step 2A, Prong Two; Step 2B: Claim 14 recite additional elements:
(a) wherein real time geo-registered information is missing information for the real time display of the virtual object in three dimensions and
(b) the non- transitory machine readable instructions that when executed by the one or more processors are further configured to
(c)generate the missing information and display a real time display of the virtual object in three dimensions on the augmented reality display system.
The additional element (a) is insignificant extra-solution activity, specifically, mere data gathering. MPEP 2106.05(g)(3), “Mere Data Gathering” examples iii, iv. The additional element is nominal or tangential addition to the claim. MPEP 2106.05(d)II examples i, iv.
Regarding (b), under BRI, could be interpreted as a computer or computer components. The computer is recited at a high level of generality, performing a generic computer function. This generic processor limitation is no more than mere instructions to apply the exception using a generic computer component. MPEP 2106.05(f).
Regarding (c), the additional element of generating/outputting/displaying step is insignificant extra-solution activity, specifically, selecting a particular data source or type of data to be manipulated. The generating/outputting/displaying step is similar to the MPEP 2106.05(g)(3), “Selecting a particular data source or type of data to be manipulated” examples iii. The generating/outputting/displaying step amounts “necessary data gathering and outputting.” MPEP 2106.05(g)(3). “Presenting Offers” and “. . . requiring a consumer to view an advertisement” MPEP 2106.05(d).II.
Claim 15
Step 1: Claim 15 depends on Claim 14 and is directed to a machine, thereby meeting step 1.
Step 2A, Prong One: Claim 15 recites additional limitations related to a judicial exception: a “mental process” abstract idea that can be performed in the human mind or by using a pen and paper and/or a mathematical concept.
simply the real time geo-registered information.
Step 2A, Prong Two; Step 2B: Claim 15 does not recite additional elements.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-9 are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Han et al. (US 2013/0131978 A1).
Regarding claim 1, Han teaches a method (Han Abstract), comprising:
receiving a data set (comprising “map data” and/or “terrain data”) (Han fig. 17 301, 304:
PNG
media_image1.png
584
310
media_image1.png
Greyscale
. “First, in the step 304, the map and terrain data is retrieved from the raw map data typically provided by a map data supplier and is converted to the format (ex. hierarchical layers and meshes) appropriate to the navigation application as shown in FIGS. 10A.” Han ¶ 0111.);
converting the data set to a common reference frame (Han fig. 17 305, 306; ¶¶ 0109-0112. In particular, Han states, “To be compatible between the terrain data and the map data so as to avoid the conventional mismatching problem such as the "dig into" or "fly above" problem arises between the roads and terrain, the process modifies the map data in view of the terrain data” (Han ¶ 0111), and “In the step 305, the map data is further modified so that the terrain data and the map data are interrelated with one another by, for example, in terms of height at locations proximate to one another. More specifically, the map data is modified so that a vertex of each road matches the corresponding height of the terrain at the most proximate location with one another so that the road follows contours of the terrain” (Han ¶ 0112). The data set are converted/modified/linked in accordance with a common reference frame based on interrelatedness/matching.); and
generating a virtual object (e.g., virtual signs or turn markers) based on the data set in the common reference frame (Han fig. 15, ¶ 0101 (“In the three-dimensional rendering environment, a plurality of three-dimensional signs and turn markers can be placed in the three-dimensional space to indicate proper routes, as they do in the real world.” The placement and type of the virtual object, e.g., sign and/or turn marker, is based on the interrelated map and terrain data.).
Regarding claim 2, Han further teaches the method of claim 1 and further teaches wherein the received data set is missing information in order to render the virtual object in three dimensions (
[BRI on the record] with respect to “missing information,” the Examiner is reading the limitation to mean: information used or needed to do something, e.g., rendering a virtual object in 3D.
[Mapping Analysis]
Han ¶¶ 0007, 0044; terrain and/or map data are missing data, because they are needed to properly render the virtual object(s), e.g., sign and/or turn marker, at a correct and meaningful location.).
Regarding claim 3, Han further teaches the method of claim 2 and further teaches the method as further comprising supplying (Han fig. 17 301, 304) the missing information (comprising “map data” and/or “terrain data”) (Han Fig. 17 301, 302; ¶¶ 0109-0110: 3D terrain data is retrieved from USGS public domain source) and rendering the virtual object in three dimensional space through an augmented reality display system (Han fig. 15; ¶ 0101: “display a three-dimensional real-time rendering environment including three-dimensional guidance information” and “a plurality of three-dimensional signs and turn markers can be placed in the three-dimensional space to indicate proper routes, as they do in the real world.” This mapping for “augmented reality” is consistent with the Spec. ¶ 0061, which includes virtual reality.).
Regarding claim 4, Han further teaches the method of claim 1 and further teaches the method as further comprising receiving a second data set (terrain data) relating to a physical environment (Han Fig. 17 301 and 302; ¶¶ 0109-0110: 3D terrain data is retrieved from USGS public domain source), wherein the first data set (map data) comprises missing information (Han Fig. 17 304; ¶ 0111: in the step 304, map data is retrieved from the raw map data typically provided by a map data supplier and is converted to the format (ex. hierarchical layers and meshes); ¶¶ 0007, 0044: explaining terrain data are missing/needed from the first data set (map data)) and the second data set comprises information related to the missing information of the first data set (Han Fig. 17 301 and 302; ¶ 0109-0110: 3D terrain data is retrieved from USGS public domain source and complement the map data. In particular, Han states, “For example, the roads may ‘dig into’ or ‘fly above’ the terrain proximate to them. In other words, if a long road segment crosses a hill or valley, it is very likely that the road no longer appears connected to the ground of the terrain. The road may be under or over the terrain surface, which may cause visual discomfort for the user.” Han ¶ 0007. Therefore, the terrain data, mapped to information missing from the map data, is not only used for virtual rendering; and it is needed to render properly.).
Regarding claim 5, Han further teaches the method of claim 4 and further teaches wherein the second data set (terrain data) comprises elevation of a ground level of the physical environment (“Typically, the converted terrain data is further converted to grayscale data with respect to each location (pixel, data point) of the terrain so that the three-dimensional image will be expressed by shadings or intensity. As is known in the art, the grayscale image carries only intensity information where the degree of intensity expresses elevation (height) of each location of the terrain.” Han ¶ 0068.).
Regarding claim 6, Han further teaches the method of claim 5 and further teaches the method as further comprising converting the second data set (terrain data) into the common reference frame (Han fig. 17 305, 306; ¶¶ 0109-0112. In particular, Han states, “To be compatible between the terrain data and the map data so as to avoid the conventional mismatching problem such as the "dig into" or "fly above" problem arises between the roads and terrain, the process modifies the map data in view of the terrain data” (Han ¶ 0111), and “In the step 305, the map data is further modified so that the terrain data and the map data are interrelated with one another by, for example, in terms of height at locations proximate to one another. More specifically, the map data is modified so that a vertex of each road matches the corresponding height of the terrain at the most proximate location with one another so that the road follows contours of the terrain” (Han ¶ 0112). The data sets, including terrain data, are converted/modified in accordance with a common reference frame based on interrelatedness/matching.).
Regarding claim 7, Han further teaches the method of claim 6 and further teaches the method as further comprising simplifying the second data set by reducing a resolution of the data set (
“In one embodiment, the step of conducting the border compensation operation includes a step of replacing the data points of a lower LOD terrain or map mesh with the data points of a corresponding higher LOD terrain or map mesh, and a step of combining the meshes of different LOD thereby stitching two or more images of different LODs.” Han ¶ 0021. Here, multiple levels of detail are generated for the second data, mapped to terrain data. The lower LOD terrain data are simplified.
“In the method for displaying three-dimensional terrains and route guidance noted above, the step of providing the map database includes steps of: configuring the map data in a multi-layered structure where a plurality of data layers with different levels of details (LODs) are arranged in a hierarchy manner where each data layer is configured by a plurality of map meshes each being defined by a plurality of data points; and modifying the map data to be compatible with the terrain data with respect to height such that a vertex of each road matches a corresponding height of the terrain at a most proximate location with one another so that a road follows contours of the terrain.” Han ¶ 0016. ).
Regarding Claim 8, Han further teaches the method of claim 7 and further teaches the method as further comprising generating a mesh mapping of ground level elevation based on the second data set (Han ¶ 0015-0016: “configuring the terrain database in a form of a plurality of terrain meshes where each terrain mesh is defined by a plurality of data points each representing a grayscale of a terrain” and “modifying the map data to be compatible with the terrain data with respect to height such that a vertex of each road matches a corresponding height of the terrain at a most proximate location with one another so that a road follows contours of the terrain.” Therefore, the contours of the terrain represents the ground level elevation.).
Regarding Claim 9, Han further teaches the method of claim 8 and further teaches the method as further comprising using the mesh mapping to supply the missing information (Han ¶ 0015-0016; ¶ 0109-0110. In particular, Han’s terrain data are to complement the map data to remedy a deficiency or missing feature, stating “For example, the roads may "dig into" or "fly above" the terrain proximate to them. In other words, if a long road segment crosses a hill or valley, it is very likely that the road no longer appears connected to the ground of the terrain. The road may be under or over the terrain surface, which may cause visual discomfort for the user.” Han ¶ 0007. Han ¶ 0015-0016: “configuring the terrain database in a form of a plurality of terrain meshes where each terrain mesh is defined by a plurality of data points each representing a grayscale of a terrain” and “modifying the map data to be compatible with the terrain data with respect to height such that a vertex of each road matches a corresponding height of the terrain at a most proximate location with one another so that a road follows contours of the terrain.” Therefore, for example, the mesh mapping for the road supplies the missing information that includes the height distribution for the road vertices.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 10 and 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Han et al. (US 2013/0131978 A1).
Regarding Claim 10, Han teaches the method of claim 1, wherein the receiving the first data set comprises having a first data set that is geo-registered (
Han Fig. 17, 304; ¶ 0111; in the step 304, map data is retrieved from the raw map data typically provided by a map data supplier and is converted to the format (ex. hierarchical layers and meshes).
The Examiner takes an Official Notice that it would have been well-known in the art that map data and terrain data may be geo-registered. The benefits of combining this well-known knowledge would have been that a user could get more accurate location/map information from the system.).
Regarding claim 13, Han teaches a system for displaying real-time, geo-registered data in an augmented reality environment (
Han Fig. 15; ¶ 0101: “display a three-dimensional real-time rendering environment including three-dimensional guidance information” and “a plurality of three-dimensional signs and turn markers can be placed in the three-dimensional space to indicate proper routes, as they do in the real world.” This mapping for “augmented reality” is consistent with the Spec. ¶ 0061.
The Examiner takes an Official Notice that it would have been well-known in the art that map data and terrain data may be geo-registered. The benefits of combining this well-known knowledge would have been that a user could get more accurate location/map information from the system.),
comprising an augmented reality display system (Han Figs. 5, 15 and 19; Abstract; ¶¶ 0101, 0028), one or more processors, memory (Han Fig 19), and non-transitory machine readable instructions that when executed by the one or more processors (Han ¶¶ 0071, 0120-0121) are configured to:
receive (Han fig. 17 301, 304) in real time geo-registered information (map and/or terrain data) (Han Fig. 17 301, 304. Han Fig. 15; ¶ 0101: “display a three-dimensional real-time rendering environment including three-dimensional guidance information” and “a plurality of three-dimensional signs and turn markers can be placed in the three-dimensional space to indicate proper routes, as they do in the real world.” In order for the map data to be displayed in real time to guide a user, the same data is received in real time before it can be displayed real time.);
transform the geo-registered information into a data set to be displayed by the augmented reality system (Han Fig. 17, 305 and 306; ¶ 0109-0112; map information (304) and terrain information (301) are set to a common reference frame and the processed data sets are stored; and the stored data will be further processed and displayed.); and display a virtual object on the augmented reality display system based on the real time geo-registered information (Han Fig. 15; ¶ 0101; In the three-dimensional rendering environment, a plurality of three-dimensional signs and turn markers can be placed in the three-dimensional space to indicate proper routes, as they do in the real world).
The Examiner takes an Official Notice that it would have been well-known in the art that map data and terrain data may be geo-registered. The benefits of combining this well-known knowledge would have been that a user could get more accurate location/map information from the system.
Regarding Claim 14, Han teaches the system of claim 13 and further teaches
wherein real time geo-registered information is missing information for the real time display of the virtual object in three dimensions (Han Fig. 17 301 and 302; ¶ 0109-0110: 3D terrain data is retrieved from USGS public domain source and complement the map data. In particular, Han states, “For example, the roads may ‘dig into’ or ‘fly above’ the terrain proximate to them. In other words, if a long road segment crosses a hill or valley, it is very likely that the road no longer appears connected to the ground of the terrain. The road may be under or over the terrain surface, which may cause visual discomfort for the user.” Han ¶ 0007.
Han ¶¶ 0007, 0044; terrain and/or map data are missing data, because they are needed to properly render the virtual object(s), e.g., sign and/or turn marker, at a correct and meaningful location.) and
the non-transitory machine readable instructions that when executed by the one or more processors are further configured to generate the missing information ( Han Fig. 17 301, 302; ¶ 0109-0110; 3D terrain data is retrieved from USGS public domain source) and display a real time display of the virtual object in three dimensions on the augmented reality display system (Han Fig. 15; ¶ 0101; display a three-dimensional real-time rendering environment including three-dimensional guidance information; a plurality of three-dimensional signs and turn markers can be placed in the three-dimensional space to indicate proper routes, as they do in the real world).
Regarding claim 15, Han teaches the system of claim 14 and further teaches wherein the non-transitory machine readable instructions that when executed by the one or more processors are further configured to simply the real time geo-registered information (
Han Fig. 17 305 306; ¶ 0109-0112; map information (304) and terrain information (301) are set to a common reference frame and the processed data sets are stored; see Han Figs. 11-14.
In particular, “In one embodiment, the step of conducting the border compensation operation includes a step of replacing the data points of a lower LOD terrain or map mesh with the data points of a corresponding higher LOD terrain or map mesh, and a step of combining the meshes of different LOD thereby stitching two or more images of different LODs.” Han ¶ 0021; also see Han ¶ 0016. Here, multiple levels of detail are generated for the data. The lower LOD terrain data are simplified.).
Claims 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over Han as applied to Claim 1, in further view of BALACHANDRESWARAN et al. (WO 2016041088 A1).
Regarding Claim 11, Han teaches the method of claim 1, and further teaches the method as further comprising
Han Fig. 15; ¶ 0101; display a three-dimensional real-time rendering environment including three-dimensional guidance information; a plurality of three-dimensional signs and turn markers can be placed in the three-dimensional space to indicate proper routes, as they do in the real world.
The Examiner takes an Official Notice that it would have been well-known in the art that map and terrain data may be geo-registered. The benefits of combining this well-known knowledge would have been that a user could get more accurate location/map information from the system.).
Han does not explicitly disclose; however, BALACHANDRESWARAN teaches receiving a location and orientation of an augmented reality system for displaying the virtual object (
BALACHANDRESWARAN states, “ . . . repeatedly and substantially continuously mapping to the virtual map the substantially real-time location and orientation of each tracked dynamic object within the physical environment; by one or more processors, for the HMD, determining the location and orientation of the field of view based on the correlation and the location and orientation of the HMD, . . ..” BALACHANDRESWARAN ¶¶ 3-6. The determined field of view further influences on how any virtual object is displayed, e.g., left-side view v. right-side view of the virtual object.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine BALACHANDRESWARAN’s HMD with Han. One of ordinary skill in the art would be motivated to allow a user to have a more immersive experience and/or to provide the user with more pertinent map or context information based on where the user is and based on the orientation of the user.
Regarding Claim 12, Han in view of BALACHANDRESWARAN teaches the method of claim 11 and further teaches the method as further comprising receiving the data set in real time for real time generation of the virtual object (Han Fig. 15; ¶ 0101: “display a three-dimensional real-time rendering environment including three-dimensional guidance information” and “a plurality of three-dimensional signs and turn markers can be placed in the three-dimensional space to indicate proper routes, as they do in the real world”).
Conclusion
The following prior art, made of record, was not relied upon but is considered pertinent to applicant's disclosure:
Menozzi et al. (US 20160055671 A1)
[0001] The described technology regards vision-aided navigation, and in particular pose estimation useful in vision aided navigation, advantageous in wearable augmented-reality (AR) systems operating in natural outdoor environments and other applications.
[0010] In horizon matching, the camera captures an image that includes the horizon, and extracts and aligns the horizon from the image with a three-dimensional height map of the surrounding terrain in the database. Thereby, the camera's absolute orientation can be measured, and this measurement is used by the processing module to update the display pose position vector.
Menozzi is similar to Applicant’s disclosed invention of an augmented reality system that is capable of being used in outdoors and the use of terrain data based on height maps. However, Menozzi appears to be vision-aided, and does not clearly delineate map data from terrain data.
US 20210125415 A1 Optimizing Head Mounted Displays for Augmented Reality – While many augmented reality systems provide “see-through” transparent or translucent displays upon which to project virtual objects, many virtual reality systems instead employ opaque, enclosed screens. Indeed, eliminating the user's perception of the real-world may be integral to some successful virtual reality experiences. Thus, head mounted displays designed exclusively for virtual reality experiences may not be easily repurposed to capture significant portions of the augmented reality market. Various of the disclosed embodiments facilitate the repurposing of a virtual reality device for augmented reality use. Particularly, by anticipating user head motion, embodiments may facilitate scene renderings better aligned with user expectations than naïve renderings generated within the enclosed field of view. In some embodiments, the system may use procedural mapping methods to generate a virtual model of the environment. The system may then use this model to supplement the anticipatory rendering. Fig. 6 is a conceptual diagram illustrating a coordinate transform representation of a pose as may be used in some embodiments.
US 20210248829 A1 Viewpoint Dependent Brick Selection for Fast Volumetric Reconstruction – A method to culling parts of a 3D reconstruction volume is provided. The method makes available to a wide variety of mobile XR applications fresh, accurate and comprehensive 3D reconstruction data with low usage of computational resources and storage spaces. The method includes culling parts of the 3D reconstruction volume against a depth image. The depth image has a plurality of pixels, each of which represents a distance to a surface in a scene. In some embodiments, the method includes culling parts of the 3D reconstruction volume against a frustum. The frustum is derived from a field of view of an image sensor, from which image data to create the 3D reconstruction is obtained. Once sufficient data from each of the devices has been gathered to identify features in a common portion of the physical world, those features may be correlated, providing the transformation from one device-specific coordinate frame to the others. One of these device-specific coordinate frames may be designated as the common coordinate frame and the transformations between the other coordinate frames, and that coordinate frame may be used to convert data from the device-specific coordinate frames to the coordinate frame designated as the common coordinate frame.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZHENGXI LIU whose telephone number is (571)270-7509. The examiner can normally be reached M-F 9 AM - 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at 571-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZHENGXI LIU/Primary Examiner, Art Unit 2611
Prosecution Insights