DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This is a non-final office action on the merits. Claims 1-6, 9-12,14,16-17, 20, 22, 26 and 28 are currently pending and are addressed below.
The examiner notes that the fundamentals of the rejection are based on the broadest reasonable interpretation of the claim language. Applicant is kindly invited to consider the reference as a whole. References are to be interpreted as by one of ordinary skill in the art rather than as by a novice. See MPEP 2141. Therefore, the relevant inquiry when interpreting a reference is not what the reference expressly discloses on its face but what the reference would teach or suggest to one of ordinary skill in the art.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 12/09/2025 has been entered.
Response to Arguments
Applicant's arguments, filed 11/17/2025, regarding the rejection of claims 1-6, 9-14, 16-17, 20, 22, 26, and 28 under 35 U.S.C 101 have been fully considered but they are not persuasive. After carefully considering applicant’s assertion that the amended claim language of the independent claims presents a viable improvement, however, the examiner respectfully disagrees. The amended limitations merely calculate image distance and adjust the displayed vertical positions of AR route guidance objects so that overlapping information is easier for the pilot to view. The examiner understands that the alleged improvement is to the presentation and human perception of information opposed to the operation or functioning of a computer, aerial vehicle, or other relevant technology in the art. As such, the claims do not appear to recite a particular technical improvement, or mechanism for improving AR technology, but instead provide a desired result of avoiding obstructing when graphical objects overlap. Accordingly, the rejection is maintained.
Applicant’s arguments, filed, 11/17/2025, with respect to claims 1, 3-6, 9-14, 16-17, 20, 22, 26, and 28 under 35 U.S.C 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations include the recitation of “image acquisition unit,” “flight route determination unit,” “object generating unit,” “AR guidance image generating unit,” “display unit,” and “event identification unit,” in claims 17, 20, 22, 26, and 28. For example, claims 17, 20, and 22-23 specifically recite “flight route determination unit.”
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6, 9-12,14,16-17, 20, 22, 26 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1
Claim 1 is directed to a method of providing augmented reality guidance for an aerial vehicle (i.e., a method). Therefore, claim 1 is within at least one of the four statutory categories..
STEP 2A (PRONG 1)
Regarding Prong I of the Step 2A analysis in the 2019 PEG, the claims are to be analyzed to determine whether they recite subject matter that falls within one of the follow groups of abstract ideas: a) mathematical concepts, b) certain methods of organizing human activity, and/or c) mental processes. Independent claim 1 includes limitations that recite an abstract idea (emphasized below) and will be used as a representative claim for the remainder of the 101 rejection.
Claim 1
A method of providing augmented reality guidance for an aerial vehicle, comprising:
acquiring an aerial vehicle flight image captured through a camera installed in the aerial vehicle
acquiring a flight route for a flight to a destination of the aerial vehicle
generating an augmented reality (AR) route guidance object corresponding to the flight route
generating an AR route guidance image by mapping the generated AR route guidance object to the aerial vehicle flight image
determining a risk of a flight route based on dynamic hazard information mapped to flight map data of the aerial vehicle
determining a risk level by applying a weight to the hazard information
displaying the generated AR route guidance image
wherein in the displaying, the AR route guidance object is displayed with different transparency and color based on the risk level in order to provide an intuitive and effective visualization of the flight route and ensure that the aerial vehicle is flying normally without deviating from a predetermined route
wherein the generating of the AR route guidance object includes: calculating an image distance to a point where a plurality of AR route guidance objects are displayed based on a viewpoint of the aerial vehicle
adjusting vertical heights of each of the plurality of AR route guidance objects according to the calculated image distance and wherein the method improves a view of a pilot of the aerial vehicle by not hindering the view when the plurality of the AR route guidance objects overlap each other
The examiner submits that the foregoing bolded limitations constitute a mental process because under its broadest reasonable interpretation, the claim covers performance of the limitations in the human mind. Highlighted above, the limitations merely consist of a process of generating data that maps additional route guidance information to an image captured by a camera, essentially providing guidance markers overlayed on the augmented image to aid the aerial vehicle. This step encompasses a person mentally mapping a planned route after analyzing information obtained from the acquire steps and planning visual aid through object markers. The generation step simply recites augmented reality at apply it level. Additionally, the “determining” steps encompasses a person looking at the dynamic hazard data collected (acquired, obtained, etc.) and forming a simple judgement of a risk and level (determination, analysis, comparison, etc.) either mentally, or using pen and paper. The recited steps of “calculating,” and “adjusting” the vertical heights merely involve a further evaluation of information and modifying a result based on that determination, or evaluation. For instance, see MPEP 2106.05(f) which provides the following considerations for determining whether a claim simply recites a judicial exception with the words “apply it” (or an equivalent), such as mere instructions to implement an abstract idea on a computer: (1) whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished; (2) whether the claim invokes computers or other machinery (in this case augmented reality) merely as a tool to perform an existing process; and (2) the particularity or generality of the application of the judicial exception. As such, claim 1 recites at least one mental process.
STEP 2A (PRONG 2)
Regarding Prong II of the Step 2A analysis in the 2019 PEG, the claims are to be analyzed to determine whether the claim, as a whole, integrates the abstract into a practical application. As noted in the 2019 PEG, it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.”
Claim 1
A method of providing augmented reality guidance for an aerial vehicle, comprising:
acquiring an aerial vehicle flight image captured through a camera installed in the aerial vehicle
acquiring a flight route for a flight to a destination of the aerial vehicle
generating an augmented reality (AR) route guidance object corresponding to the flight route
generating an AR route guidance image by mapping the generated AR route guidance object to the aerial vehicle flight image
determining a risk of a flight route based on dynamic hazard information mapped to flight map data of the aerial vehicle
determining a risk level by applying a weight to the hazard information
displaying the generated AR route guidance image
wherein in the displaying, the AR route guidance object is displayed with different transparency and color based on the risk level in order to provide an intuitive and effective visualization of the flight route and ensure that the aerial vehicle is flying normally without deviating from a predetermined route
wherein the generating of the AR route guidance object includes: calculating an image distance to a point where a plurality of AR route guidance objects are displayed based on a viewpoint of the aerial vehicle
adjusting vertical heights of each of the plurality of AR route guidance objects according to the calculated image distance and wherein the method improves a view of a pilot of the aerial vehicle by not hindering the view when the plurality of the AR route guidance objects overlap each other
For the following reasons, the examiner submits that the above identified additional limitations do not integrate the previously discussed abstract idea into a practical application. Regarding the additional limitations of, “acquiring an aerial vehicle flight image…” “acquiring a flight route…” and “displaying the generated AR route…” the examiner submits that these limitations are insignificant extra-solution activities that merely use the generic computer components to perform the processes. Specifically, the “acquiring,” steps are recited at a high level of generality (i.e. as a general means of receiving, obtaining, or acquiring information for use in the store and processing step) and amounts to mere data gathering, which is a form of insignificant extra solution activity. Additionally, the “displaying,” step is also recited at a high level of generality (i.e, as a general means of transmitting, outputting, or displaying and encompasses a post solution action, which is also a form of insignificant extra solution activity.
Thus, taken alone, the additional elements do not integrate the abstract idea into a practical application. Further, looking at the additional limitation(s) as an ordered combination or as a whole, the limitation(s) add nothing that is not already present when looking at the elements taken individually. For instance, there is no indication that the additional elements, when considered as a whole, reflect an improvement in the functioning of a computer or an improvement to another technology or technical field, apply or use the above-noted judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, implement/use the above-noted judicial exception with a particular machine or manufacture that is integral to the claim, effect a transformation or reduction of a particular article to a different state or thing, or apply or use the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is not more than a drafting effort designed to monopolize the exception (MPEP § 2106.05). Accordingly, the additional limitation(s) do/does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
STEP 2B
Regarding Step 2B of the 2019 PEG, representative independent claim 1 does not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a server to perform the generating... amounts to nothing more than applying the exception of using a generic computer component. Generally applying an exception using a generic computer component cannot provide an inventive concept.
Thus, since claims 1, 16, and 17 are: (a) directed towards abstract ideas, (b) do not recite additional elements that integrate the judicial exception into a practical application, and (c) do not recite additional elements that amount to significantly more than the judicial exception, it is clear that claim 1 is directed towards non-statutory subject matter.
As per claim 16.
Claim 16, a non-transitory recordable medium (a device), includes limitations analogous to claim 1, a method claim. Accordingly, claim 16 is rejected under 35 U.S.C 101 because the claim is directed to an abstract idea without significantly more.
As per claim 17.
Claim 17, an apparatus, includes limitations analogous to claim 1, a method claim. Accordingly, claim 17 is rejected under 35 U.S.C 101 because the claim is directed to an abstract idea without significantly more.
Dependent claims 2-6, 9-12, 14, 20, 22, 26, and 28 do not recite any further limitations that cause the claim to be patent eligible. The limitations of the dependent claims are directed towards additional aspects of the judicial exception and/or additional elements that do not integrate the judicial exception into a practical application.
Regarding some of the other examples of additional limitations in the dependent claims such as, “calculating a tilt direction…” (claim 2), “calculating a degree of deviation…” (claim 5), “determining a risk of a flight route,” (claim 6), and “generating of the AR route guidance object,” (claim 17), and “determine whether the aerial vehicle deviates…” the examiner submits that these limitations are additional abstract ideas that can be practically performed in the human mind. For example, determining steps, in the context of the claims, encompasses a person looking at data collected (acquired, obtained, etc.) and forming a simple judgement (determination, analysis, comparison, etc.) either mentally, or using a pen and paper. Similar to the analysis of claims 1, 16, and 17 the acquiring step encompasses a person mentally mapping a planned route after analyzing information obtained from the acquire steps and planning visual aid through object markers. The generation step simply recites augmented reality at apply it level.
Furthermore, some other examples of additional limitations in the dependent claims, such as, “displaying the AR route guidance…” (claim 7), “displaying an AR event guidance object,” (claim 10), and “displaying the AR route guidance object…” (claim 17) are directed towards additional aspects of the judicial exception and additional elements that do not integrate the judicial exception into a practical application. As discussed, the “displaying,” steps are recited at a high level of generality (i.e, as a general means of transmitting, outputting, or displaying and encompasses a post solution action, which is also a form of insignificant extra solution activity. The examiner submits that these elements are merely generic computer components that are meant to implement the abstract idea on a computer and merely “apply” the mental judgements in a general-purpose vehicle control environment.
As such, claims 1-6, 9-12, 14, 16-17, 20, 22, 26, and 28 are rejected under 35 U.S.C 101 as being drawn to an abstract idea without significantly more, and thus are ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-6, 9-12, 14, 16-17, 20, 22, 26, and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Yuseung Jeong et al. (KR202100114647A), hereinafter referred to as Jeong in view of Blanc Gilles et al. (US20180365265A1), hereinafter referred to as Gilles, in further view of Filliatre Eric et al. (US2010066564A1), hereinafter referred to as Eric, in further view of Ozaki Yukisuke et al. (US2013286206A1), hereinafter referred to as Yukisuke.
Regarding claim 1, Jeong discloses: a method of providing augmented reality guidance for an aerial vehicle, comprising:
acquiring an aerial vehicle flight image captured through a camera installed in the aerial vehicle (see at least Jeong, ¶¶ [0467]-[0469] which discloses flight information captured through a camera of the air vehicle, [0344] discloses a specific example of flight images of a specific region captured through the camera, which means an aerial vehicle flight image is acquired)
acquiring a flight route for a flight to a destination of the aerial vehicle (see at least Jeong, ¶¶ [0010]-[0016] which discloses acquiring location information of the destination and optimal route of the aerial vehicle via control terminal of a flight route)
generating an augmented reality (AR) route guidance object corresponding to the flight route (see at least Jeong, ¶¶ [0102] discloses the implementation of AR, [0396]-[0398] discloses the generation and processing of a virtual image overlayed along route data presented on a heads-up-display (HUD); [0455]-[0456] discloses the pattern flight item which essentially displays the expected path for the pattern flight)
generating an AR route guidance image by mapping the generated AR route guidance object to the aerial vehicle flight image (see at least Jeong, ¶¶ [0102] discloses the implementation of AR, [0396]-[0398] discloses the generation and processing of a virtual image overlayed along route data presented on a heads-up-display (HUD); [0455]-[0456] discloses the pattern flight item which that displays and maps the expected path for the pattern flight)
displaying the generated AR route guidance image (see at least Jeong, ¶¶ [0398] which discloses the displaying of the mapped virtual image containing route guidance and general flight information such as speed, altitude, direction, etc of the aerial vehicle)
Jeon is silent on, however, in the same field of endeavor, Gilles teaches: determining a risk of the flight route based on dynamic hazard information mapped to flight map data of the aerial vehicle (see at least Gilles, ¶¶ [0059], [0067] [0071], discloses the application of displaying objects through a means of augmented reality; [0108] discloses the specific calculations augmented; ¶¶ [0128]-[0130], [0140] discloses the object display of metadata representing the calculations of local/or overall risk level which may be discretized or quantified which means that the system determines a quantized risk of the overall flight route based on dynamic hazard information of individual objects mapped to the map data of a flight path)
determining risk level by applying a weight to the hazard information (see at least Gilles, ¶¶ [0059], [0067], [0071], discloses the application of displaying objects through a means of augmented reality; [0108] discloses the specific calculations augmented; ¶¶ [0128]-[0130], [0104] discloses the object display of metadata representing the calculations of local/or overall risk level which may be discretized or quantified, which means a risk level weight (quantized or threshold) is applied to the hazard information of the communicated objects regarding the flight map data of the aerial vehicle)
It would have been obvious to a person of ordinary skill in the art to modify Jeon to include determining a risk of the flight route based on dynamic hazard information mapped to flight map data of the aerial vehicle and determining risk level by applying a weight to the hazard information as taught by Gilles. Implementing the teachings into the base device of Jeon would allow for more accurate determination of hazard events posed to the aerial vehicle while traveling along a route and provide personalized guidance objects to account for longer term divergences and critical calculations to be determined in real-time.
Modified Jeon is silent on, however, in the same field of endeavor, Eric teaches: wherein in the displaying, the AR route guidance object is displayed with different transparency and color based on the risk level in order to provide an intuitive and effective visualization of the flight route and ensure that the aerial vehicle is flying normally without deviating from a predetermined route (see at least Eric, ¶¶ [0004]-[0005], which discloses an ergonomic of various graphical representations overlaid by color and transparency rules which depend on the data; [0027]-[0029], [0036], [0039]-[0042], [0050]-[0051] relates to a color code representing the risk level in order to provide an intuitive and effective visualization of the flight route and ensure that the aerial vehicle is flying normally without deviating from a predetermined route)
It would have been obvious to a person of ordinary skill in the art to further change modified Jeon to include wherein in the displaying, the AR route guidance object is displayed with different transparency and color based on the risk level in order to provide an intuitive and effective visualization of the flight route and ensure that the aerial vehicle is flying normally without deviating from a predetermined route as taught by Eric. Incorporating the teachings would allow for an improvement to the base invention of modified Jeon that enables the pilot to navigate with knowledge of the environment via an ergonomic of various graphical representations overlaid by color and transparency which depend on the correlated data.
Further modified Jeon is silent on, however, in the same field of endeavor, Yukisuke teaches: wherein the generating of the AR route guidance object includes: calculating an image distance to a point where a plurality of AR route guidance objects are displayed based on a viewpoint of the aerial vehicle (see at least Yukisuke, ¶¶ [0096]-[0100], which discloses defining the image objective capturing range based on its angle view and optical axis, determining the icon’s position in the AR image based on the vehicle direction; vertical position is determined based on the standard line and distance from the vehicle location to the posted location)
and adjusting vertical heights of each of the plurality of AR route guidance objects according to the calculated image distance (see at least Yukisuke, ¶¶ [0100]-[0101], which discloses adjusting an upward shift of an icon based on distance, this means adjusting vertical heights of each of the plurality of AR route guidance objects according to the calculated image distance)
and wherein the method improves a view of a pilot of the aerial vehicle by not hindering the view when the plurality of the AR route guidance objects overlap each other (see at least Yukisuke, ¶¶ [0101], which discloses that the upward displacement according to distance of an icon secures visibility even when a plurality of icons overlap one another, this means and wherein the method improves a view of a pilot of the aerial vehicle by not hindering the view when the plurality of the AR route guidance objects overlap each other)
It would have been obvious to a person of ordinary skill in the art to further change modified Jeon to include wherein the AR guidance object generation calculates an image distance to a point where a plurality of AR route guidance objects are displayed based on the viewpoint of the aerial vehicle, and adjusts vertical heights of each of the plurality of AR route guidance objects according to the calculated image distance, and wherein the apparatus improves a view of a pilot of the aerial vehicle by not hindering the view when the plurality of the AR route guidance objects overlap each other. Incorporating the teachings would allow for an improvement of visibility of multiple icons that orient themselves according to calculated image distance.
Regarding claim 3, Jeong discloses: the method of claim 1, wherein the AR route guidance image further includes an object indicating a slope of the aerial vehicle in yaw, pitch, and roll directions (see at least Jeong, ¶¶ [0055]-[0057] discloses the tilt calculations of the aerial vehicles, roll, pitch, and yaw (Euler angles) through the sensors of the vehicle, [0398] which discloses the displaying of the mapped virtual image containing route guidance and general flight information such as speed, altitude, direction, etc of the aerial vehicle))
Regarding claim 4, Jeong discloses: the method of claim 1, further comprising:
comparing a location of the aerial vehicle with the flight route to determine whether the aerial vehicle deviates from the route wherein, in the displaying, the AR route guidance object is displayed differently depending on whether the aerial vehicle deviates from the route (see at least Jeong, ¶¶ [0023] discloses the display and setting of waypoints an aerial vehicle will travel along, essentially the current route location and in the event that there is an obstacle, determines a detour route that deviates from the original waypoints; transmission of data includes the amount of change in the azimuth and creation of changing points which deviates from the waypoints; [0024]-[0026] discloses the determination of a vehicle flying through inflection points that deviate from a route)
Regarding claim 5, Jeong discloses: the method of claim 4, further comprising calculating a degree of the deviation from the route when the aerial vehicle deviates from the route, wherein, in the displaying, the AR route guidance object is displayed differently depending on the degree of the deviation from the route (see at least Jeong, ¶¶ [0023] discloses the setting of waypoints an aerial vehicle will travel along, essentially the current route location and in the event that there is an obstacle, determines a detour route that deviates from the original waypoints; display of data includes the amount of change in the azimuth and creation of changing points which deviates from the waypoints; [0024]-[0026] discloses the determination of a vehicle flying through inflection points that deviate from a route)
Regarding claim 6, Jeong discloses: the method of claim 1, wherein the dynamic hazard information includes at least one of weather information, bird flock information, and other aerial vehicles information (see at least Jeong, ¶¶ [0370]-[0373] discloses recognition of risk when flying along a route indicative of obstacles and/or landmarks on the route; [0391] discloses the recognition of other aerial vehicle information such as presence (positional information) according to current route where a vehicle can determine if there is a risk of colliding with another vehicle)
Regarding claim 9, Jeong discloses: the method of claim 1, further comprising, when an altitude of the aerial vehicle is out of a reference altitude range, generating an AR altitude danger guidance object indicating altitude danger, wherein, in the displaying, the generated AR altitude danger guidance object is displayed on the AR route guidance image (see at least Jeong, ¶¶ [0237] discloses height reporting based on the event that the vehicle’s altitude exceeds a network referenced threshold; [0397]-[0398] discloses the virtual image overlayed on the flight image including information such as altitude)
Regarding claim 10, Jeong discloses: the method of claim 1, further comprising, when an event is detected from the aerial vehicle flight image during the flight of the aerial vehicle, identifying a type of the event, wherein, in the displaying, an AR event guidance object indicating the event according to the identified type of the event is displayed on the AR route guidance image (see at least Jeong, ¶¶ [0391] discloses the recognition of a collision event according to current route where a vehicle can determine if there is a risk of colliding with another vehicle; [0393]-[0397] discloses the transmission of a passage interval (sets a distance and direction) determined by the device to avoid collision which can be displayed as a virtual image overlayed along a route)
Regarding claim 11, Jeong discloses: the method of claim 10, wherein the event includes at least one of a bird flock event, a collision risk building, a vertiport, and a prohibited area (see at least Jeong, ¶¶ [0391] discloses the recognition of a collision event according to current route where a vehicle can determine if there is a risk of colliding with another vehicle; [0393]-[0397] discloses the transmission of a passage interval (sets a distance and direction) determined by the device to avoid collision which can be displayed as a virtual image overlayed along a route)
Regarding claim 12, Jeong discloses: the method of claim 1, wherein, in the generating of the AR route guidance object, an AR route guidance object composed of a plurality of objects is generated, and the AR route guidance object is generated by adjusting an arrangement interval of the plurality of objects according to whether the route is a curve route or a straight route (see at least Jeong, ¶¶ [0434] discloses the drawing of a gentle curve along a route, overlaying and adjusting three sequentially connected inflection points to deviate from a straight route (generation of a plurality of objects – points) to efficiently perform a task, Fig.15 discloses an example of the generation of inflection points to transition from a straight route)
Regarding claim 14, Jeong discloses: the method of claim 1, wherein the AR route guidance image includes at least one of:
a first AR route guidance image displaying the AR route guidance object on a forward image that is transmitted through a windshield of the aerial vehicle and shown to a passenger (see at least Jeong, ¶¶ [0102] discloses the displaying of virtual images/data transmission through various mediums such as a glass type terminal of the vehicle, a head mounted display worn by a passenger, etc.)
a second AR route guidance image displaying the AR route guidance object in the captured aerial vehicle flight image shown to the passenger through a screen (see at least Jeong, ¶¶ [0102] discloses the displaying of virtual images/data transmission through various mediums such as a glass type terminal of the vehicle, a head mounted display worn by a passenger, etc.)
Regarding claim 16, Jeong discloses: a non-transitory computer-readable recording medium (see at least Jeong, ¶¶ [0102]) in which a program for executing a method of providing augmented reality guidance for an aerial vehicle, wherein the method comprising:
acquiring an aerial vehicle flight image captured through a camera installed in the aerial vehicle (see at least Jeong, ¶¶ [0467]-[0469] which discloses flight information captured through a camera of the air vehicle, [0344] discloses a specific example of flight images of a specific region captured through the camera)
acquiring a flight route for a flight to a destination of the aerial vehicle (see at least Jeong, ¶¶ [0010]-[0016] which discloses location information of the destination and optimal route of the aerial vehicle via control terminal)
generating an augmented reality (AR) route guidance object corresponding to the flight route (see at least Jeong, ¶¶ [0102] discloses the implementation of AR, [0396]-[0398] discloses the generation and processing of a virtual image overlayed along route data presented on a heads-up-display (HUD); [0455]-[0456] discloses the pattern flight item which essentially displays the expected path for the pattern flight)
generating an AR route guidance image by mapping the generated AR route guidance object to the aerial vehicle flight image (see at least Jeong, ¶¶ [0102] discloses the implementation of AR, [0396]-[0398] discloses the generation and processing of a virtual image overlayed along route data presented on a heads-up-display (HUD); [0455]-[0456] discloses the pattern flight item which that displays and maps the expected path for the pattern flight)
displaying the generated AR route guidance image (see at least Jeong, ¶¶ [0398] which discloses the displaying of the mapped virtual image containing route guidance and general flight information such as speed, altitude, direction, etc of the aerial vehicle)
Jeon is silent on, however, in the same field of endeavor, Gilles teaches: determining a risk of the flight route based on dynamic hazard information mapped to flight map data of the aerial vehicle (see at least Gilles, ¶¶ [0059], [0067] [0071], discloses the application of displaying objects through a means of augmented reality; [0108] discloses the specific calculations augmented; ¶¶ [0128]-[0130], [0140] discloses the object display of metadata representing the calculations of local/or overall risk level which may be discretized or quantified which means that the system determines a quantized risk of the overall flight route based on dynamic hazard information of individual objects mapped to the map data of a flight path)
determining risk level by applying a weight to the hazard information (see at least Gilles, ¶¶ [0059], [0067], [0071], discloses the application of displaying objects through a means of augmented reality; [0108] discloses the specific calculations augmented; ¶¶ [0128]-[0130], [0104] discloses the object display of metadata representing the calculations of local/or overall risk level which may be discretized or quantified, which means a risk level weight (quantized or threshold) is applied to the hazard information of the communicated objects regarding the flight map data of the aerial vehicle)
It would have been obvious to a person of ordinary skill in the art to modify Jeon to include determining a risk of the flight route based on dynamic hazard information mapped to flight map data of the aerial vehicle and determining risk level by applying a weight to the hazard information as taught by Gilles. Implementing the teachings into the base device of Jeon would allow for more accurate determination of hazard events posed to the aerial vehicle while traveling along a route and provide personalized guidance objects to account for longer term divergences and critical calculations to be determined in real-time.
Modified Jeon is silent on, however, in the same field of endeavor, Eric teaches: wherein in the displaying, the AR route guidance object is displayed with different transparency and color based on the risk level in order to provide an intuitive and effective visualization of the flight route and ensure that the aerial vehicle is flying normally without deviating from a predetermined route (see at least Eric, ¶¶ [0004]-[0005], which discloses an ergonomic of various graphical representations overlaid by color and transparency rules which depend on the data; [0027]-[0029], [0036], [0039]-[0042], [0050]-[0051] relates to a color code representing the risk level in order to provide an intuitive and effective visualization of the flight route and ensure that the aerial vehicle is flying normally without deviating from a predetermined route)
It would have been obvious to a person of ordinary skill in the art to further change modified Jeon to include wherein in the displaying, the AR route guidance object is displayed with different transparency and color based on the risk level in order to provide an intuitive and effective visualization of the flight route and ensure that the aerial vehicle is flying normally without deviating from a predetermined route as taught by Eric. Incorporating the teachings would allow for an improvement to the base invention of modified Jeon that enables the pilot to navigate with knowledge of the environment via an ergonomic of various graphical representations overlaid by color and transparency which depend on the correlated data.
Further modified Jeon is silent on, however, in the same field of endeavor, Yukisuke teaches: wherein the generating of the AR route guidance object includes: calculating an image distance to a point where a plurality of AR route guidance objects are displayed based on a viewpoint of the aerial vehicle (see at least Yukisuke, ¶¶ [0096]-[0100], which discloses defining the image objective capturing range based on its angle view and optical axis, determining the icon’s position in the AR image based on the vehicle direction; vertical position is determined based on the standard line and distance from the vehicle location to the posted location)
and adjusting vertical heights of each of the plurality of AR route guidance objects according to the calculated image distance (see at least Yukisuke, ¶¶ [0100]-[0101], which discloses adjusting an upward shift of an icon based on distance, this means adjusting vertical heights of each of the plurality of AR route guidance objects according to the calculated image distance)
and wherein the method improves a view of a pilot of the aerial vehicle by not hindering the view when the plurality of the AR route guidance objects overlap each other (see at least Yukisuke, ¶¶ [0101], which discloses that the upward displacement according to distance of an icon secures visibility even when a plurality of icons overlap one another, this means and wherein the method improves a view of a pilot of the aerial vehicle by not hindering the view when the plurality of the AR route guidance objects overlap each other)
It would have been obvious to a person of ordinary skill in the art to further change modified Jeon to include wherein the AR guidance object generation calculates an image distance to a point where a plurality of AR route guidance objects are displayed based on the viewpoint of the aerial vehicle, and adjusts vertical heights of each of the plurality of AR route guidance objects according to the calculated image distance, and wherein the apparatus improves a view of a pilot of the aerial vehicle by not hindering the view when the plurality of the AR route guidance objects overlap each other. Incorporating the teachings would allow for an improvement of visibility of multiple icons that orient themselves according to calculated image distance.
Regarding claim 17, Jeong discloses: an apparatus for providing augmented reality guidance (see at least Jeong, ¶¶ [0007]), comprising:
an image acquisition unit installed in an aerial vehicle to acquire a flight image of the aerial vehicle (see at least Jeong, ¶¶ [0467]-[0469] which discloses flight information captured through a camera of the air vehicle, [0344] discloses a specific example of flight images of a specific region captured through the camera)
a flight route determination unit generating a flight route for a flight to a destination of the aerial vehicle (see at least Jeong, ¶¶ [0010]-[0016] which discloses location information of the destination and optimal route of the aerial vehicle via control terminal)
an AR guidance object generating unit generating an augmented reality (AR) route guidance object corresponding to the flight route (see at least Jeong, ¶¶ [0102] discloses the implementation of AR, [0396]-[0398] discloses the generation and processing of a virtual image overlayed along route data presented on a heads-up-display (HUD); [0455]-[0456] discloses the pattern flight item which essentially displays the expected path for the pattern flight)
an AR guidance image generation unit generating an AR route guidance image by mapping the generated AR route guidance object to the aerial vehicle flight image (see at least Jeong, ¶¶ [0102] discloses the implementation of AR, [0396]-[0398] discloses the generation and processing of a virtual image overlayed along route data presented on a heads-up-display (HUD); [0455]-[0456] discloses the pattern flight item which that displays and maps the expected path for the pattern flight)
and a display unit displaying the generated AR route guidance image (see at least Jeong, ¶¶ [0398] which discloses the displaying of the mapped virtual image containing route guidance and general flight information such as speed, altitude, direction, etc of the aerial vehicle)
Jeon is silent on, however, in the same field of endeavor, Gilles teaches:
wherein the flight route determination unit determines the risk of the flight route based on dynamic hazard information mapped to flight map data of the aerial vehicle (see at least Gilles, ¶¶ [0059], [0067] [0071], discloses the application of displaying objects through a means of augmented reality; [0108] discloses the specific calculations augmented; ¶¶ [0128]-[0130], [0140] discloses the object display of metadata representing the calculations of local/or overall risk level which may be discretized or quantified which means that the system determines a quantized risk of the overall flight route based on dynamic hazard information of individual objects mapped to the map data of a flight path)
determines a risk level by applying a weight to the hazard information (see at least Gilles, ¶¶ [0059], [0067], [0071], discloses the application of displaying objects through a means of augmented reality; [0108] discloses the specific calculations augmented; ¶¶ [0128]-[0130], [0104] discloses the object display of metadata representing the calculations of local/or overall risk level which may be discretized or quantified, which means a risk level weight (quantized or threshold) is applied to the hazard information of the communicated objects regarding the flight map data of the aerial vehicle)
It would have been obvious to a person of ordinary skill in the art to modify Jeon to include determining a risk of the flight route based on dynamic hazard information mapped to flight map data of the aerial vehicle and determining risk level by applying a weight to the hazard information as taught by Gilles. Implementing the teachings into the base device of Jeon would allow for more accurate determination of hazard events posed to the aerial vehicle while traveling along a route and provide personalized guidance objects to account for longer term divergences and critical calculations to be determined in real-time. Modified Jeon is silent on, however, in the same field of endeavor, Eric teaches: the display unit displays the AR route guidance object with different transparency and color based on the risk level in order to provide an intuitive and effective visualization of the flight route and ensure that the aerial vehicle is flying normally without deviating from a predetermined route (see at least Eric, ¶¶ [0004]-[0005], which discloses an ergonomic of various graphical representations overlaid by color and transparency rules which depend on the data; [0027]-[0029], [0036], [0039]-[0042], [0050]-[0051] relates to a color code representing the risk level in order to provide an intuitive and effective visualization of the flight route and ensure that the aerial vehicle is flying normally without deviating from a predetermined route)
It would have been obvious to a person of ordinary skill in the art to further change modified Jeon to include the display unit displays the AR route guidance object with different transparency and color based on the risk level in order to provide an intuitive and effective visualization of the flight route and ensure that the aerial vehicle is flying normally without deviating from a predetermined route as taught by Eric. Incorporating the teachings would allow for an improvement to the base invention of modified Jeon that enables the pilot to navigate with knowledge of the environment via an ergonomic of various graphical representations overlaid by color and transparency which depend on the correlated data.
Further modified Jeon is silent on, however, in the same field of endeavor, Yukisuke teaches: wherein the AR guidance object generation calculates an image distance to a point where a plurality of AR route guidance objects are displayed based on the viewpoint of the aerial vehicle (see at least Yukisuke, ¶¶ [0096]-[0100], which discloses defining the image objective capturing range based on its angle view and optical axis, determining the icon’s position in the AR image based on the vehicle direction; vertical position is determined based on the standard line and distance from the vehicle location to the posted location)
and adjusts vertical heights of each of the plurality of AR route guidance objects according to the calculated image distance (see at least Yukisuke, ¶¶ [0100]-[0101], which discloses adjusting an upward shift of an icon based on distance, this means adjusting vertical heights of each of the plurality of AR route guidance objects according to the calculated image distance)
and wherein the apparatus improves a view of a pilot of the aerial vehicle by not hindering the view when the plurality of the AR route guidance objects overlap each other (see at least Yukisuke, ¶¶ [0101], which discloses that the upward displacement according to distance of an icon secures visibility even when a plurality of icons overlap one another, this means and wherein the method improves a view of a pilot of the aerial vehicle by not hindering the view when the plurality of the AR route guidance objects overlap each other)
It would have been obvious to a person of ordinary skill in the art to further change modified Jeon to include wherein the AR guidance object generation calculates an image distance to a point where a plurality of AR route guidance objects are displayed based on the viewpoint of the aerial vehicle, and adjusts vertical heights of each of the plurality of AR route guidance objects according to the calculated image distance, and wherein the apparatus improves a view of a pilot of the aerial vehicle by not hindering the view when the plurality of the AR route guidance objects overlap each other. Incorporating the teachings would allow for an improvement of visibility of multiple icons that orient themselves according to calculated image distance.
Regarding claim 20, Jeong discloses: the apparatus of claim 17, wherein the flight route determination unit compares a location of the aerial vehicle with the flight route to determine whether the aerial vehicle deviates from the route, and the display unit displays the AR route guidance object differently depending on whether the aerial vehicle deviates from the route (see at least Jeong, ¶¶ [0023] discloses the display and setting of waypoints an aerial vehicle will travel along, essentially the current route location and in the event that there is an obstacle, determines a detour route that deviates from the original waypoints; transmission of data includes the amount of change in the azimuth and creation of changing points which deviates from the waypoints; [0024]-[0026] discloses the determination of a vehicle flying through inflection points that deviate from a route)
Regarding claim 22, Jeong discloses: the apparatus of claim 17, wherein the dynamic hazard information includes at least one of weather information, bird flock information, and other aerial vehicles information (see at least Jeong, ¶¶ [0370]-[0373] discloses recognition of risk when flying along a route indicative of obstacles and/or landmarks on the route; [0391] discloses the recognition of other aerial vehicle information such as presence (positional information) according to current route where a vehicle can determine if there is a risk of colliding with another vehicle)
Regarding claim 26, Jeong discloses: the apparatus of claim 17, further comprising, when an event is detected from the aerial vehicle flight image during the flight of the aerial vehicle, an event identification unit identifies a type of the event, wherein the display unit displays an AR event guidance object indicating the event according to the identified type of the event on the AR route guidance image (see at least Jeong, ¶¶ [0391] discloses the recognition of a collision event according to current route where a vehicle can determine if there is a risk of colliding with another vehicle; [0393]-[0397] discloses the transmission of a passage interval (sets a distance and direction) determined by the device to avoid collision which can be displayed as a virtual image overlayed along a route)
Regarding claim 28, Jeong discloses: the apparatus of claim 17, wherein the AR guidance object generation unit generates an AR route guidance object composed of a plurality of objects, and generates the AR route guidance object by adjusting an arrangement interval of the plurality of objects according to whether the route is a curve route or a straight route (see at least Jeong, ¶¶ [0434] discloses the drawing of a gentle curve along a route, overlaying and adjusting three sequentially connected inflection points to deviate from a straight route (generation of a plurality of objects – points) to efficiently perform a task, Fig.15 discloses an example of the generation of inflection points to transition from a straight route)
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over further modified Jeong in view of Mehta Ankit et al. (US20210195103A1), hereinafter referred to as Ankit.
Regarding claim 2, Jeong discloses:
The method of claim 1, wherein the acquiring of the aerial vehicle flight image includes: calculating a tilt direction slope value during the flight of the aerial vehicle (see at least Jeong, ¶¶ [0055]-[0057] discloses the tilt calculations of the aerial vehicles, roll, pitch, and yaw (Euler angles) through the sensors of the vehicle)
Jeong is silent, however, Ankit discloses:
correcting a tilt direction slope of the camera so that the camera keeps level based on the calculated slope value (see at least Ankit ¶¶ [0014]-[0015], [0083]-[0084] which discloses the ability to direct the optical axis of the camera of the aerial vehicle in different directions according to different values; [0032]-[0033] discloses the ability for the camera to be reoriented in the direction perpendicular to the first axis of yaw movement to eliminate the need to pan the camera to stay level; [0061]-[0062] discloses the ability of the camera to be oriented along tilt movement)
It would have been obvious to a person of ordinary skill in the art to modify Jeong to include correcting a tilt direction slope of the camera so that the camera keeps level based on the calculated slope value as taught by Ankit. Doing so would allow for an improvement technique to the base device of Jeong which already accounts for factors such as aerial vehicle yaw, pitch, and roll values. In the embodiment of Jeong, images of the flight route are acquired through the installed camera and it is already provided that the camera can be adjusted in terms of aperture of and magnification. Implementing the teaching of Ankit ensures that the camera is able to orient itself according to the known Euler values and accurately acquire images without having to pan. In general, this eliminates unidirectional distortion as a vehicle moves along a route.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KIRSTEN JADE M SANTOS whose telephone number is (571)272-7442. The examiner can normally be reached Monday: 8:00 am - 4:00 pm, 6:00-8:00 pm (+ with flex).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rachid Bendidi can be reached at (571) 272-4896. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KIRSTEN JADE M SANTOS/Examiner, Art Unit 3664
/RACHID BENDIDI/Supervisory Patent Examiner, Art Unit 3664
Prosecution Insights