DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 10-14 are rejected under 35 U.S.C. 103 as being unpatentable over Menozzi et al. (US Pub. 2016/0055671), hereinafter Menozzi, in view of Schmitz et al. (US Pub. 2012/0022820), hereinafter Schmitz.
Regarding claim 10, Menozzi discloses a sports mask comprising: a head support structure that is adapted to be connected to or supported on a head of a user, wherein the head support structure is a portion of a snorkel, scuba unit, or full face sports mask (Fig. 1C; Paragraph [0025]: motion sensing and visualization kit 1 may include a rigid reference assembly 11 with a camera (high-speed and high-resolution) 12 and a sensor package 13, and having a display 14 with a graphical user interface 141 rendered on the display to convey information in the form of text and graphics, an example of which is shown in FIGS. 1A, 1B, 1C, 2 and 9. In some embodiments of the system of the described technology the display 14 is see-through. The sensors and processing module of the described technology can function with a wide variety of displays, including by example and without limitation see-through displays manufactured by the BAE, Lumus, and SA Photonics; Paragraph [0057]: In head-worn applications, the location of the sensor package on the motion sensing and visualization kit, and the corresponding kinematics due to head movement, result in angular rate being a very good indicator; Paragraph [0095]: immersive training may be extended to sports training applications, where athletes may use augmented and/or virtual reality to enhance their training program); a lens that is supported on the head support structure and is sufficiently transparent to allow the user to see therethrough into a real world environment (Fig. 1A; Paragraph [0025]: motion sensing and visualization kit 1 may include a rigid reference assembly 11 with a camera (high-speed and high-resolution) 12 and a sensor package 13, and having a display 14 with a graphical user interface 141 rendered on the display to convey information in the form of text and graphics, an example of which is shown in FIGS. 1A, 1B, 1C, 2 and 9. In some embodiments of the system of the described technology the display 14 is see-through. The sensors and processing module of the described technology can function with a wide variety of displays, including by example and without limitation see-through displays manufactured by the BAE, Lumus, and SA Photonics); and a system that is structured and configured to: (1) capture one or more images of one or more visual features from the real world environment (Fig. 7; Paragraph [0066]: point s′ is the point in the undistorted camera image corresponding to the real-world reference point s; Paragraph [0095]: Outdoor recreational enthusiasts (e.g., runners, bicyclists, hikers) can be presented with directional information, waypoints, and details about their exact position and heading when carrying out active movement while in a heads-up posture viewing the real-world environment. For immersive training applications, users can be presented with virtual avatars that appear as part of their real-world environment as they maneuver and carry out training scenarios indoors or outdoors. Such training enables the user to practice and improve scenario-specific decision making. This immersive training may be extended to sports training applications, where athletes may use augmented and/or virtual reality to enhance their training program. The pose estimation systems and methods herein may also be applied to gaming scenarios where augmented reality and/or virtual reality is used to enhance user experience and the estimation of pose of a gaming device is required. Other applications include the transportation industry, where vehicle operators may access information that appears to part of the real-world environment, and maintenance personnel may view pertinent information overlaid on the system under maintenance/repair); (2) identify the one or more visual features from the real world environment using the one or more captured images (Fig. 5; Fig. 7; Paragraph [0003]): processing module also is communicatively coupled with and uses information from a database, which receives, stores, and transmits data such as locations of interest and objects in an environment in geodetic coordinates (latitude, longitude, altitude), and digital terrain and elevation data. Geo-spatial data relating to locations of interest and objects in the environment may be transferred to and from the database by means of a radio. The database may be local to the processing module, virtual, or stored in a single or network of remote servers on the Internet or otherwise accessible to the processing module; Paragraph [0067]: Absolute orientation may be achieved by the processing module 2 of the described technology executing methods of landmark matching, horizon matching, and Sun-matching. The landmark matching (LM) method uses imagery from the camera 12 to track the location of a distant object of known coordinates and provide a measurement of orientation to the EKF. Prior to operation, the user must select a feature in the environment (i.e., a landmark) that can be visually recognized during operation and whose coordinates are known by the system (stored in the database); (3) calculate a path through the real world environment based upon the one or more identified features from the real world environment (Paragraphs [0032]-[0033]: processing module 2 or other computation means receives measurements from the sensors 13, and calculates over time the position and orientation of the display (fixedly coupled with the camera and the sensors), and the position vector of a location of interest s relative to the spatial position of the display 14 vector of the location of interest s so calculated by the processing module is referred to as vector rdsd, representing the current estimated position of s relative to the position of the display (such as the orientation, geodetic position (longitude, latitude, altitude), or combination thereof), expressed in display coordinates. The display pose estimations are based upon an Extended Kalman Filter (EKF) structure, performed by an “EKF Calculator” stored in memory and executable by a processor to calculate state predictions; Paragraph [0067]: the user must select a feature in the environment (i.e., a landmark) that can be visually recognized during operation and whose coordinates are known by the system (stored in the database); Paragraph [0095]: Outdoor recreational enthusiasts (e.g., runners, bicyclists, hikers) can be presented with directional information, waypoints, and details about their exact position and heading when carrying out active movement while in a heads-up posture viewing the real-world environment. For immersive training applications, users can be presented with virtual avatars that appear as part of their real-world environment as they maneuver and carry out training scenarios indoors or outdoors. Such training enables the user to practice and improve scenario-specific decision making. This immersive training may be extended to sports training applications, where athletes may use augmented and/or virtual reality to enhance their training program. The pose estimation systems and methods herein may also be applied to gaming scenarios where augmented reality and/or virtual reality is used to enhance user experience and the estimation of pose of a gaming device is required. Other applications include the transportation industry, where vehicle operators may access information that appears to part of the real-world environment, and maintenance personnel may view pertinent information overlaid on the system under maintenance/repair); and a translucent image onto the lens in orientation of a line-of-sight of the user to provide a visual navigation to the user (Fig. 9; Paragraphs [0004]-[0005]: data received by the processing module may include time-stamped sensor values including camera imagery, accelerometer measurements, rate-gyro measurements, magnetometer measurements, barometric pressure sensor measurements, GPS receiver position measurements, and GPS receiver velocity measurements. From this data and the data representing the spatial position and orientation of the display relative to the spatial position of the sensors, the processing module calculates over time a position vector representing the current estimated position of a location of interest with respect to the display, expressed in display coordinates. In some embodiments the position vector derives from the sensor values received by the processing module, using an Extended Kalman Filter (EKF) structure, adjusted to compensate for magnetic model bias based on absolute azimuth measurements received from at least one vision-aiding process, for example by means of circuitry, firmware and/or a processor. The processing module then generates signals to render on a display, at a position derived from the position vector, graphical user interface information including a graphical icon representing the location of interest. The processing module may implement a baseline GPS/INS, modified by absolute orientation information when available…plurality of sensors or sensor package may include sensors such as a camera, a 3-axis accelerometer, a 3-axis angular rate gyro, a 3-axis magnetometer, a barometric pressure sensor, and a GPS receiver, and may be mounted to a rigid reference assembly. Data from the sensor package, which could include delayed measurements, is transmitted to the processing module or other computation means, which generates signals that render graphical user interface information on a display using the sensor data. The processing module also is communicatively coupled with, and uses information from a database, which receives, stores, and transmits data such as locations of interest in geodetic coordinates (latitude, longitude, altitude), and digital terrain and elevation data. The database may be local to the processing module, virtual, stored on a single or network of remote servers, on the Internet or otherwise accessible to the processing module; Paragraph [0077]: facilitate data compression and improve processing efficiency, a continuous connected chain is created that represents the 360-degree horizon silhouette. First, edges are extracted from the projected spherical image followed by a known edge-following computation in the image to define an edge chain. While the edge chain is a very good representation of the horizon, it is also a very dense representation posing computational challenges for the alignment. This leads to a second step in which the pixel-resolution chain is reduced to a much smaller set of line segments that satisfy a maximum tangential distance. The resulting piece-wise linear representation typically reduces the complexity of the horizon and greatly boosts the computational efficiency; Paragraph [0096]: the position and orientation of the smartphone or tablet is accurately tracked while the user holds the device in an upright position in their line of sight to view the real-world while ‘looking through the phone’. In this video see-through application, geo-registered graphical information is superimposed on the device's camera imagery and presented to the user real-time on the device's display).
Menozzi does not explicitly disclose (4) project the calculated path onto the lens so as to overlay the actual features of the real environment.
However, Schmitz teaches positional navigation display (Abstract), further comprising (4) project the calculated path onto the lens so as to overlay the actual features of the real environment (Paragraph [0078]: the method in accordance with the invention is expediently useable in combination with a graphic display, on which the direction and the distance to the reference points is displayed. The position of the reference points as well as the previously dived path can be displayed on a map-like illustration. Corresponding depth information relative to the reference points or to the path can also be displayed. The path or the reference points can thereby also be shown in different colors depending on depth, so that the display remains easy to read, but still provides additional navigation information to the diver). Schmitz teaches that this will allow for accuracy in determining of position (Abstract). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Menozzi with the features of above as taught by Schmitz so as to allow for accuracy in determining of position as presented by Schmitz.
Regarding claim 11, Menozzi, in view of Schmitz teaches the sports mask defined in claim 10 Menozzi discloses wherein the system is structured and configured to calculate an enhanced relative earth-centered, earth-fixed, three-dimension location fix that is calibrated to absolute earth-centered, earth-fixed coordinates and to project that location fix onto the lens (Fig. 4; Paragraph [0041]: Various coordinate systems are involved in the calculations of the described technology, as depicted in FIG. 4. The body coordinate system b is the reference for the motion sensing and visualization kit 1, with origin at the point p. The camera coordinate system c consists of permutation of the body coordinate system's axes and shares the same origin. The display coordinate system d and the accelerometer coordinate system a are both rigidly attached to the body coordinate system. Coordinate system a is the reference for the motion sensing and visualization kit's sensor package 13. Coordinate system n is the North-East-Down (NED) reference for navigation. The Earth-Centered Earth-Fixed (ECEF) coordinate system e is used to specify points in the environment. Coordinate system i is the Earth Centered Inertial (ECI) coordinate system, which is a good approximation of a true inertial reference in the context of the described technology. The WGS-84 ellipsoid is used as the world model).
Regarding claim 12, Menozzi, in view of Schmitz teaches the sports mask defined in claim 11 Menozzi discloses wherein the system is structured and configured to additionally calculate an orientation of a line-of-sight lens of the user relative to the location of the sports mask in two axis of azimuth and altitude and to project that orientation onto the lens (Fig. 4; Paragraph [0031]: the magnetometer 133 provides a measure ym of the Earth's magnetic field to help in determining azimuth, and the barometric pressure sensor 134 provides a measure ybp of barometric pressure for estimating altitude. Similarly, the GPS receiver 135 provides its position data yGp (latitude, longitude, and altitude) and its velocity data yGv (North, East, and Down velocities). The camera 12 captures and transmits to the processing module 2 digital imagery that is processed by the processing module to provide vision-based information useful to calculate absolute orientation; Paragraph [0096]: the position and orientation of the smartphone or tablet is accurately tracked while the user holds the device in an upright position in their line of sight to view the real-world while ‘looking through the phone’. In this video see-through application, geo-registered graphical information is superimposed on the device's camera imagery and presented to the user real-time on the device's display).
Regarding claim 13, Menozzi, in view of Schmitz teaches the sports mask defined in claim 10 Menozzi discloses wherein the system is structured and configured to identify the one or more visual features from the real world environment using detected and extracted edges and features from the real world environment (Paragraphs [0074]-[0077]: horizon matching (HM) method provides a measurement of absolute orientation by comparing edges detected in the camera imagery with a horizon silhouette edge generated from DTED, using a hierarchical search computation initialized at the current orientation estimate from the EKF. In contrast to traditional horizon-matching models, the method of the described technology uses real-world data and can generate refined orientation measurements at 20 Hz with current hardware. Further the methods of the described technology overcome problems with the prior art which seek only a strong-gradient edge in an image, not necessarily present under all lighting conditions and in cases of severe occlusions by foreground object…edges are extracted from the projected spherical image followed by a known edge-following computation in the image to define an edge chain. While the edge chain is a very good representation of the horizon, it is also a very dense representation posing computational challenges for the alignment. This leads to a second step in which the pixel-resolution chain is reduced to a much smaller set of line segments that satisfy a maximum tangential distance. The resulting piece-wise linear representation typically reduces the complexity of the horizon and greatly boosts the computational efficiency).
Regarding claim 14, Menozzi, in view of Schmitz teaches the sports mask defined in claim 13 Menozzi discloses wherein the system is structured and configured to calculate the path through the real world environment by computing three dimension vectors based on changes in the movement of the features from the real world environment (Paragraphs [0032]-[0033]: processing module 2 or other computation means receives measurements from the sensors 13, and calculates over time the position and orientation of the display (fixedly coupled with the camera and the sensors), and the position vector of a location of interest s relative to the spatial position of the display 14 vector of the location of interest s so calculated by the processing module is referred to as vector rdsd, representing the current estimated position of s relative to the position of the display (such as the orientation, geodetic position (longitude, latitude, altitude), or combination thereof), expressed in display coordinates. The display pose estimations are based upon an Extended Kalman Filter (EKF) structure, performed by an “EKF Calculator” stored in memory and executable by a processor to calculate state predictions; Paragraph [0095]: Outdoor recreational enthusiasts (e.g., runners, bicyclists, hikers) can be presented with directional information, waypoints, and details about their exact position and heading when carrying out active movement while in a heads-up posture viewing the real-world environment. For immersive training applications, users can be presented with virtual avatars that appear as part of their real-world environment as they maneuver and carry out training scenarios indoors or outdoors. Such training enables the user to practice and improve scenario-specific decision making. This immersive training may be extended to sports training applications, where athletes may use augmented and/or virtual reality to enhance their training program. The pose estimation systems and methods herein may also be applied to gaming scenarios where augmented reality and/or virtual reality is used to enhance user experience and the estimation of pose of a gaming device is required. Other applications include the transportation industry, where vehicle operators may access information that appears to part of the real-world environment, and maintenance personnel may view pertinent information overlaid on the system under maintenance/repair).
Claims 1 and 3-9 are rejected under 35 U.S.C. 103 as being unpatentable over Menozzi, in view of Schmitz, and further in view of Park (US Pub. 2007/0064311).
Regarding claim 1, Menozzi discloses a snorkeling and diving sports mask for use in navigation in an underwater environment comprising: a head support structure that is adapted to be connected to or supported on a head of a user in in an underwater environment (Fig. 1C; Paragraph [0025]: motion sensing and visualization kit 1 may include a rigid reference assembly 11 with a camera (high-speed and high-resolution) 12 and a sensor package 13, and having a display 14 with a graphical user interface 141 rendered on the display to convey information in the form of text and graphics, an example of which is shown in FIGS. 1A, 1B, 1C, 2 and 9. In some embodiments of the system of the described technology the display 14 is see-through. The sensors and processing module of the described technology can function with a wide variety of displays, including by example and without limitation see-through displays manufactured by the BAE, Lumus, and SA Photonics; Paragraph [0057]: In head-worn applications, the location of the sensor package on the motion sensing and visualization kit, and the corresponding kinematics due to head movement, result in angular rate being a very good indicator; Paragraph [0095]: immersive training may be extended to sports training applications, where athletes may use augmented and/or virtual reality to enhance their training program); a lens that is supported on the head support structure and is sufficiently transparent to allow the user to see therethrough into a real world underwater environment (Fig. 1A; Paragraph [0025]: motion sensing and visualization kit 1 may include a rigid reference assembly 11 with a camera (high-speed and high-resolution) 12 and a sensor package 13, and having a display 14 with a graphical user interface 141 rendered on the display to convey information in the form of text and graphics, an example of which is shown in FIGS. 1A, 1B, 1C, 2 and 9. In some embodiments of the system of the described technology the display 14 is see-through. The sensors and processing module of the described technology can function with a wide variety of displays, including by example and without limitation see-through displays manufactured by the BAE, Lumus, and SA Photonics); and a system that is structured and configured to: (1) capture one or more images of one or more visual features from the real world underwater environment (Fig. 7; Paragraph [0066]: point s′ is the point in the undistorted camera image corresponding to the real-world reference point s; Paragraph [0095]: Outdoor recreational enthusiasts (e.g., runners, bicyclists, hikers) can be presented with directional information, waypoints, and details about their exact position and heading when carrying out active movement while in a heads-up posture viewing the real-world environment. For immersive training applications, users can be presented with virtual avatars that appear as part of their real-world environment as they maneuver and carry out training scenarios indoors or outdoors. Such training enables the user to practice and improve scenario-specific decision making. This immersive training may be extended to sports training applications, where athletes may use augmented and/or virtual reality to enhance their training program. The pose estimation systems and methods herein may also be applied to gaming scenarios where augmented reality and/or virtual reality is used to enhance user experience and the estimation of pose of a gaming device is required. Other applications include the transportation industry, where vehicle operators may access information that appears to part of the real-world environment, and maintenance personnel may view pertinent information overlaid on the system under maintenance/repair); (2) identify the one or more visual features from the real world underwater environment using the one or more captured images (Fig. 5; Fig. 7; Paragraph [0003]): processing module also is communicatively coupled with and uses information from a database, which receives, stores, and transmits data such as locations of interest and objects in an environment in geodetic coordinates (latitude, longitude, altitude), and digital terrain and elevation data. Geo-spatial data relating to locations of interest and objects in the environment may be transferred to and from the database by means of a radio. The database may be local to the processing module, virtual, or stored in a single or network of remote servers on the Internet or otherwise accessible to the processing module; Paragraph [0067]: Absolute orientation may be achieved by the processing module 2 of the described technology executing methods of landmark matching, horizon matching, and Sun-matching. The landmark matching (LM) method uses imagery from the camera 12 to track the location of a distant object of known coordinates and provide a measurement of orientation to the EKF. Prior to operation, the user must select a feature in the environment (i.e., a landmark) that can be visually recognized during operation and whose coordinates are known by the system (stored in the database); (3) calculate a path through the real world underwater environment based upon the one or more identified features from the real world underwater environment (Paragraphs [0032]-[0033]: processing module 2 or other computation means receives measurements from the sensors 13, and calculates over time the position and orientation of the display (fixedly coupled with the camera and the sensors), and the position vector of a location of interest s relative to the spatial position of the display 14 vector of the location of interest s so calculated by the processing module is referred to as vector rdsd, representing the current estimated position of s relative to the position of the display (such as the orientation, geodetic position (longitude, latitude, altitude), or combination thereof), expressed in display coordinates. The display pose estimations are based upon an Extended Kalman Filter (EKF) structure, performed by an “EKF Calculator” stored in memory and executable by a processor to calculate state predictions; Paragraph [0067]: the user must select a feature in the environment (i.e., a landmark) that can be visually recognized during operation and whose coordinates are known by the system (stored in the database); Paragraph [0095]: Outdoor recreational enthusiasts (e.g., runners, bicyclists, hikers) can be presented with directional information, waypoints, and details about their exact position and heading when carrying out active movement while in a heads-up posture viewing the real-world environment. For immersive training applications, users can be presented with virtual avatars that appear as part of their real-world environment as they maneuver and carry out training scenarios indoors or outdoors. Such training enables the user to practice and improve scenario-specific decision making. This immersive training may be extended to sports training applications, where athletes may use augmented and/or virtual reality to enhance their training program. The pose estimation systems and methods herein may also be applied to gaming scenarios where augmented reality and/or virtual reality is used to enhance user experience and the estimation of pose of a gaming device is required. Other applications include the transportation industry, where vehicle operators may access information that appears to part of the real-world environment, and maintenance personnel may view pertinent information overlaid on the system under maintenance/repair).
Menozzi does not explicitly disclose wherein the real world environment is a real world underwater environment; or (4) project the calculated path onto the lens so as to overlay the actual features of the real underwater environment to provide a visual navigation to the user in the underwater environment.
However, Schmitz teaches positional navigation display (Abstract), further comprising wherein the real world environment is a real world underwater environment (Paragraph [0001]: invention relates to a method of inertial navigation under water, in particular for scuba divers); and (4) project the calculated path onto the lens so as to overlay the actual features of the real underwater environment to provide a visual navigation to the user in the underwater environment (Paragraph [0078]: the method in accordance with the invention is expediently useable in combination with a graphic display, on which the direction and the distance to the reference points is displayed. The position of the reference points as well as the previously dived path can be displayed on a map-like illustration. Corresponding depth information relative to the reference points or to the path can also be displayed. The path or the reference points can thereby also be shown in different colors depending on depth, so that the display remains easy to read, but still provides additional navigation information to the diver). Schmitz teaches that this will allow for accuracy in determining of position (Abstract). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Menozzi with the features of above as taught by Schmitz so as to allow for accuracy in determining of position as presented by Schmitz.
Menozzi, in view of Schmitz does not explicitly disclose on a head of a user in a watertight manner.
However, Park teaches a head-mounted display (Abstract), further comprising on a head of a user in a watertight manner (Paragraph [0073]: display system 100 is waterproof (e.g., submersible in water or other liquids). For example, projector head 104, mirror 106, and/or screen 108 may use water tight seals so that water is inhibited from entering the internals of the components. Screen 108 and/or mirror 106 may be designed as a sealed face mask similar to a diving mask or scuba mask. A cover or other device may be used to allow a path for the images to project through display system 100 while the system is submerged). Park teaches that this will allow for a person to be submerged in water (Paragraph [0073]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Menozzi, in view of Schmitz with the features of above as taught by Park so as to allow for a person to be submerged in water as presented by Park.
Regarding claim 3, Menozzi, in view of Schmitz, and further in view of Park teaches the sports mask defined in claim 1, Park discloses wherein the head support structure is a portion of a scuba unit (Paragraph [0073]: display system 100 is waterproof (e.g., submersible in water or other liquids). For example, projector head 104, mirror 106, and/or screen 108 may use water tight seals so that water is inhibited from entering the internals of the components. Screen 108 and/or mirror 106 may be designed as a sealed face mask similar to a diving mask or scuba mask).
Regarding claim 4, Menozzi, in view of Schmitz, and further in view of Park teaches the sports mask defined in claim 1, Menozzi discloses wherein the head support structure is a portion of a full face sports mask (Paragraph [0057]: In head-worn applications, the location of the sensor package on the motion sensing and visualization kit, and the corresponding kinematics due to head movement, result in angular rate being a very good indicator; Paragraph [0095]: immersive training may be extended to sports training applications, where athletes may use augmented and/or virtual reality to enhance their training program).
Regarding claim 5, Menozzi, in view of Schmitz, and further in view of Park teaches the sports mask defined in claim 1, Menozzi discloses wherein the system is structured and configured to collect location, orientation, and visual information from the real world environment, calculate the path through the real world environment based upon the collect location, orientation, and visual information from the real world environment, and project the projected path as a translucent image onto the lens in orientation of a line-of-sight of the user (Fig. 9; Paragraphs [0004]-[0005]: data received by the processing module may include time-stamped sensor values including camera imagery, accelerometer measurements, rate-gyro measurements, magnetometer measurements, barometric pressure sensor measurements, GPS receiver position measurements, and GPS receiver velocity measurements. From this data and the data representing the spatial position and orientation of the display relative to the spatial position of the sensors, the processing module calculates over time a position vector representing the current estimated position of a location of interest with respect to the display, expressed in display coordinates. In some embodiments the position vector derives from the sensor values received by the processing module, using an Extended Kalman Filter (EKF) structure, adjusted to compensate for magnetic model bias based on absolute azimuth measurements received from at least one vision-aiding process, for example by means of circuitry, firmware and/or a processor. The processing module then generates signals to render on a display, at a position derived from the position vector, graphical user interface information including a graphical icon representing the location of interest. The processing module may implement a baseline GPS/INS, modified by absolute orientation information when available…plurality of sensors or sensor package may include sensors such as a camera, a 3-axis accelerometer, a 3-axis angular rate gyro, a 3-axis magnetometer, a barometric pressure sensor, and a GPS receiver, and may be mounted to a rigid reference assembly. Data from the sensor package, which could include delayed measurements, is transmitted to the processing module or other computation means, which generates signals that render graphical user interface information on a display using the sensor data. The processing module also is communicatively coupled with, and uses information from a database, which receives, stores, and transmits data such as locations of interest in geodetic coordinates (latitude, longitude, altitude), and digital terrain and elevation data. The database may be local to the processing module, virtual, stored on a single or network of remote servers, on the Internet or otherwise accessible to the processing module; Paragraph [0077]: facilitate data compression and improve processing efficiency, a continuous connected chain is created that represents the 360-degree horizon silhouette. First, edges are extracted from the projected spherical image followed by a known edge-following computation in the image to define an edge chain. While the edge chain is a very good representation of the horizon, it is also a very dense representation posing computational challenges for the alignment. This leads to a second step in which the pixel-resolution chain is reduced to a much smaller set of line segments that satisfy a maximum tangential distance. The resulting piece-wise linear representation typically reduces the complexity of the horizon and greatly boosts the computational efficiency; Paragraph [0096]: the position and orientation of the smartphone or tablet is accurately tracked while the user holds the device in an upright position in their line of sight to view the real-world while ‘looking through the phone’. In this video see-through application, geo-registered graphical information is superimposed on the device's camera imagery and presented to the user real-time on the device's display).
Regarding claim 6, Menozzi, in view of Schmitz, and further in view of Park teaches the sports mask defined in claim 1, Menozzi discloses wherein the system is structured and configured to calculate an enhanced relative earth-centered, earth-fixed, three-dimension location fix that is calibrated to absolute earth-centered, earth-fixed coordinates and to project that location fix onto the lens (Fig. 4; Paragraph [0041]: Various coordinate systems are involved in the calculations of the described technology, as depicted in FIG. 4. The body coordinate system b is the reference for the motion sensing and visualization kit 1, with origin at the point p. The camera coordinate system c consists of permutation of the body coordinate system's axes and shares the same origin. The display coordinate system d and the accelerometer coordinate system a are both rigidly attached to the body coordinate system. Coordinate system a is the reference for the motion sensing and visualization kit's sensor package 13. Coordinate system n is the North-East-Down (NED) reference for navigation. The Earth-Centered Earth-Fixed (ECEF) coordinate system e is used to specify points in the environment. Coordinate system i is the Earth Centered Inertial (ECI) coordinate system, which is a good approximation of a true inertial reference in the context of the described technology. The WGS-84 ellipsoid is used as the world model).
Regarding claim 7, Menozzi, in view of Schmitz, and further in view of Park teaches the sports mask defined in claim 6, Menozzi discloses wherein the system is structured and configured to additionally calculate an orientation of a line-of-sight lens of the user relative to the location of the sports mask in two axis of azimuth and altitude and to project that orientation onto the lens (Fig. 4; Paragraph [0031]: the magnetometer 133 provides a measure ym of the Earth's magnetic field to help in determining azimuth, and the barometric pressure sensor 134 provides a measure ybp of barometric pressure for estimating altitude. Similarly, the GPS receiver 135 provides its position data yGp (latitude, longitude, and altitude) and its velocity data yGv (North, East, and Down velocities). The camera 12 captures and transmits to the processing module 2 digital imagery that is processed by the processing module to provide vision-based information useful to calculate absolute orientation; Paragraph [0096]: the position and orientation of the smartphone or tablet is accurately tracked while the user holds the device in an upright position in their line of sight to view the real-world while ‘looking through the phone’. In this video see-through application, geo-registered graphical information is superimposed on the device's camera imagery and presented to the user real-time on the device's display).
Regarding claim 8, Menozzi, in view of Schmitz, and further in view of Park teaches the sports mask defined in claim 1, Menozzi discloses wherein the system is structured and configured to identify the one or more visual features from the real world environment using detected and extracted edges and features from the real world environment (Paragraphs [0074]-[0077]: horizon matching (HM) method provides a measurement of absolute orientation by comparing edges detected in the camera imagery with a horizon silhouette edge generated from DTED, using a hierarchical search computation initialized at the current orientation estimate from the EKF. In contrast to traditional horizon-matching models, the method of the described technology uses real-world data and can generate refined orientation measurements at 20 Hz with current hardware. Further the methods of the described technology overcome problems with the prior art which seek only a strong-gradient edge in an image, not necessarily present under all lighting conditions and in cases of severe occlusions by foreground object…edges are extracted from the projected spherical image followed by a known edge-following computation in the image to define an edge chain. While the edge chain is a very good representation of the horizon, it is also a very dense representation posing computational challenges for the alignment. This leads to a second step in which the pixel-resolution chain is reduced to a much smaller set of line segments that satisfy a maximum tangential distance. The resulting piece-wise linear representation typically reduces the complexity of the horizon and greatly boosts the computational efficiency).
Regarding claim 9, Menozzi, in view of Schmitz, and further in view of Park teaches the sports mask defined in claim 8, Menozzi discloses wherein the system is structured and configured to calculate the path through the real world environment by computing three dimension vectors based on changes in the movement of the features from the real world environment (Paragraphs [0032]-[0033]: processing module 2 or other computation means receives measurements from the sensors 13, and calculates over time the position and orientation of the display (fixedly coupled with the camera and the sensors), and the position vector of a location of interest s relative to the spatial position of the display 14 vector of the location of interest s so calculated by the processing module is referred to as vector rdsd, representing the current estimated position of s relative to the position of the display (such as the orientation, geodetic position (longitude, latitude, altitude), or combination thereof), expressed in display coordinates. The display pose estimations are based upon an Extended Kalman Filter (EKF) structure, performed by an “EKF Calculator” stored in memory and executable by a processor to calculate state predictions; Paragraph [0095]: Outdoor recreational enthusiasts (e.g., runners, bicyclists, hikers) can be presented with directional information, waypoints, and details about their exact position and heading when carrying out active movement while in a heads-up posture viewing the real-world environment. For immersive training applications, users can be presented with virtual avatars that appear as part of their real-world environment as they maneuver and carry out training scenarios indoors or outdoors. Such training enables the user to practice and improve scenario-specific decision making. This immersive training may be extended to sports training applications, where athletes may use augmented and/or virtual reality to enhance their training program. The pose estimation systems and methods herein may also be applied to gaming scenarios where augmented reality and/or virtual reality is used to enhance user experience and the estimation of pose of a gaming device is required. Other applications include the transportation industry, where vehicle operators may access information that appears to part of the real-world environment, and maintenance personnel may view pertinent information overlaid on the system under maintenance/repair).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Menozzi, in view of Schmitz, and further in view of Eisenhardt et al. (US Pub. 2012/0235902), hereinafter Eisenhardt.
Regarding claim 2, Menozzi, in view of Schmitz teaches the sports mask defined in claim 1.
Menozzi, in view of Schmitz, and further in view of Park does not explicitly disclose wherein the head support structure is a portion of a snorkel.
However, Eisenhardt teaches a head-mounted navigation display (Abstract), further comprising wherein the head support structure is a portion of a snorkel (Fig. 1C; Paragraph [0041]: FIG. 1C shows an underwater mask 10C incorporating a head-mounted information system 10C' according to a particular example embodiment of the invention. Mask 10C may have the features of traditional underwater mask for a SCUBA diver or snorkeler, for example. Mask 10C includes processing circuitry configured to implement systems and methods according to example embodiments of the invention, as discussed below. In the illustrated embodiment, mask 10C comprises a frame 12C which has openings for receiving lens assemblies 11C and 11C'. In other embodiments, mask 10C could comprise a single lens assembly. Lens assemblies 11C and 11C' may be coated with an anti-fog sealant. Either or both of the lens assemblies 11C and 11C' may be formed to define a recess in order to fit around a display unit 60 (discussed further below). Display unit 60 may be coupled to frame 12C so as to be positioned below a user's right eye when mask 10C is worn, or at any other convenient location, as discussed further below). Eisenhardt teaches that this is a traditional feature of an underwater mask for a diver (Paragraph [0041]) and Examiner notes that a snorkel is well known in the art of diving. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Menozzi, in view of Schmitz, and further in view of Park with the features of wherein the head support structure is a portion of a snorkel as presented by Eisenhardt.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW D SALVUCCI whose telephone number is (571)270-5748. The examiner can normally be reached M-F: 7:30-4:00PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, XIAO WU can be reached at (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW SALVUCCI/Primary Examiner, Art Unit 2613
Prosecution Insights